[jira] [Commented] (LUCENE-5002) Deadlock in DocumentsWriterFlushControl
[
https://issues.apache.org/jira/browse/LUCENE-5002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13660669#comment-13660669
]
Sergiusz Urbaniak commented on LUCENE-5002:
---
Simon,
No FileNotFoundExceptions what so ever. The stack trace above is complete
except the crappy ejb stack forrest which is not relevant.
Again thanks for the quick reaction, we'll use deleteQuery(new
MatchAllDocsQuery()) instead and omit the global lock.
Deadlock in DocumentsWriterFlushControl
---
Key: LUCENE-5002
URL: https://issues.apache.org/jira/browse/LUCENE-5002
Project: Lucene - Core
Issue Type: Bug
Affects Versions: 4.3
Environment: OpenJDK 64-Bit Server VM (23.7-b01 mixed mode)
Linux Ubuntu Server 12.04 LTS 64-Bit
Reporter: Sergiusz Urbaniak
Assignee: Simon Willnauer
Fix For: 5.0, 4.4, 4.3.1
Attachments: LUCENE-5002.patch, LUCENE-5002_test.patch
Hi all,
We have an obvious deadlock between a MaybeRefreshIndexJob thread
calling ReferenceManager.maybeRefresh(ReferenceManager.java:204) and a
RebuildIndexJob thread calling
IndexWriter.deleteAll(IndexWriter.java:2065).
Lucene wants to flush in the MaybeRefreshIndexJob thread trying to
intrinsically lock the IndexWriter instance at
{{DocumentsWriterPerThread.java:563}} before notifyAll()ing the flush.
Simultaneously the RebuildIndexJob thread who already intrinsically locked
the IndexWriter instance at IndexWriter#deleteAll wait()s at
{{DocumentsWriterFlushControl.java:245}} for the flush forever causing a
deadlock.
{code}
MaybeRefreshIndexJob Thread - 2 daemon prio=10 tid=0x7f8fe4006000
nid=0x1ac2 waiting for monitor entry [0x7f8fa7bf7000]
java.lang.Thread.State: BLOCKED (on object monitor)
at
org.apache.lucene.index.IndexWriter.useCompoundFile(IndexWriter.java:2223)
- waiting to lock 0xf1c00438 (a
org.apache.lucene.index.IndexWriter)
at
org.apache.lucene.index.DocumentsWriterPerThread.sealFlushedSegment(DocumentsWriterPerThread.java:563)
at
org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:533)
at
org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:422)
at
org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:559)
at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:365)
- locked 0xf1c007d0 (a java.lang.Object)
at
org.apache.lucene.index.StandardDirectoryReader.doOpenFromWriter(StandardDirectoryReader.java:270)
at
org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:245)
at
org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:235)
at
org.apache.lucene.index.DirectoryReader.openIfChanged(DirectoryReader.java:170)
at
org.apache.lucene.search.SearcherManager.refreshIfNeeded(SearcherManager.java:118)
at
org.apache.lucene.search.SearcherManager.refreshIfNeeded(SearcherManager.java:58)
at
org.apache.lucene.search.ReferenceManager.doMaybeRefresh(ReferenceManager.java:155)
at
org.apache.lucene.search.ReferenceManager.maybeRefresh(ReferenceManager.java:204)
at jobs.MaybeRefreshIndexJob.timeout(MaybeRefreshIndexJob.java:47)
RebuildIndexJob Thread - 1 prio=10 tid=0x7f903000a000 nid=0x1a38 in
Object.wait() [0x7f9037dd6000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on 0xf1c0c240 (a
org.apache.lucene.index.DocumentsWriterFlushControl)
at java.lang.Object.wait(Object.java:503)
at
org.apache.lucene.index.DocumentsWriterFlushControl.waitForFlush(DocumentsWriterFlushControl.java:245)
- locked 0xf1c0c240 (a
org.apache.lucene.index.DocumentsWriterFlushControl)
at
org.apache.lucene.index.DocumentsWriter.abort(DocumentsWriter.java:235)
- locked 0xf1c05370 (a
org.apache.lucene.index.DocumentsWriter)
at org.apache.lucene.index.IndexWriter.deleteAll(IndexWriter.java:2065)
- locked 0xf1c00438 (a org.apache.lucene.index.IndexWriter)
at jobs.RebuildIndexJob.buildIndex(RebuildIndexJob.java:102)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (LUCENE-5002) Deadlock in DocumentsWriterFlushControl
Sergiusz Urbaniak created LUCENE-5002:
-
Summary: Deadlock in DocumentsWriterFlushControl
Key: LUCENE-5002
URL: https://issues.apache.org/jira/browse/LUCENE-5002
Project: Lucene - Core
Issue Type: Bug
Affects Versions: 4.3
Environment: OpenJDK 64-Bit Server VM (23.7-b01 mixed mode)
Linux Ubuntu Server 12.04 LTS 64-Bit
Reporter: Sergiusz Urbaniak
Hi all,
We have an obvious deadlock between a MaybeRefreshIndexJob thread
calling ReferenceManager.maybeRefresh(ReferenceManager.java:204) and a
RebuildIndexJob thread calling
IndexWriter.deleteAll(IndexWriter.java:2065).
Lucene wants to flush in the MaybeRefreshIndexJob thread trying to
intrinsically lock the IndexWriter instance at
{{DocumentsWriterPerThread.java:563}} before notifyAll()ing the flush.
Simultaneously the RebuildIndexJob thread who already intrinsically locked
the IndexWriter instance at IndexWriter#deleteAll wait()s at
{{DocumentsWriterFlushControl.java:245}} for the flush forever causing a
deadlock.
{code}
MaybeRefreshIndexJob Thread - 2 daemon prio=10 tid=0x7f8fe4006000
nid=0x1ac2 waiting for monitor entry [0x7f8fa7bf7000]
java.lang.Thread.State: BLOCKED (on object monitor)
at
org.apache.lucene.index.IndexWriter.useCompoundFile(IndexWriter.java:2223)
- waiting to lock 0xf1c00438 (a
org.apache.lucene.index.IndexWriter)
at
org.apache.lucene.index.DocumentsWriterPerThread.sealFlushedSegment(DocumentsWriterPerThread.java:563)
at
org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:533)
at
org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:422)
at
org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:559)
at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:365)
- locked 0xf1c007d0 (a java.lang.Object)
at
org.apache.lucene.index.StandardDirectoryReader.doOpenFromWriter(StandardDirectoryReader.java:270)
at
org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:245)
at
org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:235)
at
org.apache.lucene.index.DirectoryReader.openIfChanged(DirectoryReader.java:170)
at
org.apache.lucene.search.SearcherManager.refreshIfNeeded(SearcherManager.java:118)
at
org.apache.lucene.search.SearcherManager.refreshIfNeeded(SearcherManager.java:58)
at
org.apache.lucene.search.ReferenceManager.doMaybeRefresh(ReferenceManager.java:155)
at
org.apache.lucene.search.ReferenceManager.maybeRefresh(ReferenceManager.java:204)
at jobs.MaybeRefreshIndexJob.timeout(MaybeRefreshIndexJob.java:47)
RebuildIndexJob Thread - 1 prio=10 tid=0x7f903000a000 nid=0x1a38 in
Object.wait() [0x7f9037dd6000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on 0xf1c0c240 (a
org.apache.lucene.index.DocumentsWriterFlushControl)
at java.lang.Object.wait(Object.java:503)
at
org.apache.lucene.index.DocumentsWriterFlushControl.waitForFlush(DocumentsWriterFlushControl.java:245)
- locked 0xf1c0c240 (a
org.apache.lucene.index.DocumentsWriterFlushControl)
at
org.apache.lucene.index.DocumentsWriter.abort(DocumentsWriter.java:235)
- locked 0xf1c05370 (a
org.apache.lucene.index.DocumentsWriter)
at org.apache.lucene.index.IndexWriter.deleteAll(IndexWriter.java:2065)
- locked 0xf1c00438 (a org.apache.lucene.index.IndexWriter)
at jobs.RebuildIndexJob.buildIndex(RebuildIndexJob.java:102)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5002) Deadlock in DocumentsWriterFlushControl
[
https://issues.apache.org/jira/browse/LUCENE-5002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13659343#comment-13659343
]
Sergiusz Urbaniak commented on LUCENE-5002:
---
Implementation note: we (obviously) use the same IndexWriter instance across
all threads.
Deadlock in DocumentsWriterFlushControl
---
Key: LUCENE-5002
URL: https://issues.apache.org/jira/browse/LUCENE-5002
Project: Lucene - Core
Issue Type: Bug
Affects Versions: 4.3
Environment: OpenJDK 64-Bit Server VM (23.7-b01 mixed mode)
Linux Ubuntu Server 12.04 LTS 64-Bit
Reporter: Sergiusz Urbaniak
Hi all,
We have an obvious deadlock between a MaybeRefreshIndexJob thread
calling ReferenceManager.maybeRefresh(ReferenceManager.java:204) and a
RebuildIndexJob thread calling
IndexWriter.deleteAll(IndexWriter.java:2065).
Lucene wants to flush in the MaybeRefreshIndexJob thread trying to
intrinsically lock the IndexWriter instance at
{{DocumentsWriterPerThread.java:563}} before notifyAll()ing the flush.
Simultaneously the RebuildIndexJob thread who already intrinsically locked
the IndexWriter instance at IndexWriter#deleteAll wait()s at
{{DocumentsWriterFlushControl.java:245}} for the flush forever causing a
deadlock.
{code}
MaybeRefreshIndexJob Thread - 2 daemon prio=10 tid=0x7f8fe4006000
nid=0x1ac2 waiting for monitor entry [0x7f8fa7bf7000]
java.lang.Thread.State: BLOCKED (on object monitor)
at
org.apache.lucene.index.IndexWriter.useCompoundFile(IndexWriter.java:2223)
- waiting to lock 0xf1c00438 (a
org.apache.lucene.index.IndexWriter)
at
org.apache.lucene.index.DocumentsWriterPerThread.sealFlushedSegment(DocumentsWriterPerThread.java:563)
at
org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:533)
at
org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:422)
at
org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:559)
at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:365)
- locked 0xf1c007d0 (a java.lang.Object)
at
org.apache.lucene.index.StandardDirectoryReader.doOpenFromWriter(StandardDirectoryReader.java:270)
at
org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:245)
at
org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:235)
at
org.apache.lucene.index.DirectoryReader.openIfChanged(DirectoryReader.java:170)
at
org.apache.lucene.search.SearcherManager.refreshIfNeeded(SearcherManager.java:118)
at
org.apache.lucene.search.SearcherManager.refreshIfNeeded(SearcherManager.java:58)
at
org.apache.lucene.search.ReferenceManager.doMaybeRefresh(ReferenceManager.java:155)
at
org.apache.lucene.search.ReferenceManager.maybeRefresh(ReferenceManager.java:204)
at jobs.MaybeRefreshIndexJob.timeout(MaybeRefreshIndexJob.java:47)
RebuildIndexJob Thread - 1 prio=10 tid=0x7f903000a000 nid=0x1a38 in
Object.wait() [0x7f9037dd6000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on 0xf1c0c240 (a
org.apache.lucene.index.DocumentsWriterFlushControl)
at java.lang.Object.wait(Object.java:503)
at
org.apache.lucene.index.DocumentsWriterFlushControl.waitForFlush(DocumentsWriterFlushControl.java:245)
- locked 0xf1c0c240 (a
org.apache.lucene.index.DocumentsWriterFlushControl)
at
org.apache.lucene.index.DocumentsWriter.abort(DocumentsWriter.java:235)
- locked 0xf1c05370 (a
org.apache.lucene.index.DocumentsWriter)
at org.apache.lucene.index.IndexWriter.deleteAll(IndexWriter.java:2065)
- locked 0xf1c00438 (a org.apache.lucene.index.IndexWriter)
at jobs.RebuildIndexJob.buildIndex(RebuildIndexJob.java:102)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5002) Deadlock in DocumentsWriterFlushControl
[
https://issues.apache.org/jira/browse/LUCENE-5002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13659508#comment-13659508
]
Sergiusz Urbaniak commented on LUCENE-5002:
---
Hi,
Thanks for the quick feedback! As long as the sync issues on IW are unresolved
we declare IW instances as *not thread-safe* for our development and
synchronize access to it externally (of course as mentioned in the docs not on
the IW instance itself).
Deadlock in DocumentsWriterFlushControl
---
Key: LUCENE-5002
URL: https://issues.apache.org/jira/browse/LUCENE-5002
Project: Lucene - Core
Issue Type: Bug
Affects Versions: 4.3
Environment: OpenJDK 64-Bit Server VM (23.7-b01 mixed mode)
Linux Ubuntu Server 12.04 LTS 64-Bit
Reporter: Sergiusz Urbaniak
Assignee: Simon Willnauer
Attachments: LUCENE-5002_test.patch
Hi all,
We have an obvious deadlock between a MaybeRefreshIndexJob thread
calling ReferenceManager.maybeRefresh(ReferenceManager.java:204) and a
RebuildIndexJob thread calling
IndexWriter.deleteAll(IndexWriter.java:2065).
Lucene wants to flush in the MaybeRefreshIndexJob thread trying to
intrinsically lock the IndexWriter instance at
{{DocumentsWriterPerThread.java:563}} before notifyAll()ing the flush.
Simultaneously the RebuildIndexJob thread who already intrinsically locked
the IndexWriter instance at IndexWriter#deleteAll wait()s at
{{DocumentsWriterFlushControl.java:245}} for the flush forever causing a
deadlock.
{code}
MaybeRefreshIndexJob Thread - 2 daemon prio=10 tid=0x7f8fe4006000
nid=0x1ac2 waiting for monitor entry [0x7f8fa7bf7000]
java.lang.Thread.State: BLOCKED (on object monitor)
at
org.apache.lucene.index.IndexWriter.useCompoundFile(IndexWriter.java:2223)
- waiting to lock 0xf1c00438 (a
org.apache.lucene.index.IndexWriter)
at
org.apache.lucene.index.DocumentsWriterPerThread.sealFlushedSegment(DocumentsWriterPerThread.java:563)
at
org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:533)
at
org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:422)
at
org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:559)
at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:365)
- locked 0xf1c007d0 (a java.lang.Object)
at
org.apache.lucene.index.StandardDirectoryReader.doOpenFromWriter(StandardDirectoryReader.java:270)
at
org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:245)
at
org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:235)
at
org.apache.lucene.index.DirectoryReader.openIfChanged(DirectoryReader.java:170)
at
org.apache.lucene.search.SearcherManager.refreshIfNeeded(SearcherManager.java:118)
at
org.apache.lucene.search.SearcherManager.refreshIfNeeded(SearcherManager.java:58)
at
org.apache.lucene.search.ReferenceManager.doMaybeRefresh(ReferenceManager.java:155)
at
org.apache.lucene.search.ReferenceManager.maybeRefresh(ReferenceManager.java:204)
at jobs.MaybeRefreshIndexJob.timeout(MaybeRefreshIndexJob.java:47)
RebuildIndexJob Thread - 1 prio=10 tid=0x7f903000a000 nid=0x1a38 in
Object.wait() [0x7f9037dd6000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on 0xf1c0c240 (a
org.apache.lucene.index.DocumentsWriterFlushControl)
at java.lang.Object.wait(Object.java:503)
at
org.apache.lucene.index.DocumentsWriterFlushControl.waitForFlush(DocumentsWriterFlushControl.java:245)
- locked 0xf1c0c240 (a
org.apache.lucene.index.DocumentsWriterFlushControl)
at
org.apache.lucene.index.DocumentsWriter.abort(DocumentsWriter.java:235)
- locked 0xf1c05370 (a
org.apache.lucene.index.DocumentsWriter)
at org.apache.lucene.index.IndexWriter.deleteAll(IndexWriter.java:2065)
- locked 0xf1c00438 (a org.apache.lucene.index.IndexWriter)
at jobs.RebuildIndexJob.buildIndex(RebuildIndexJob.java:102)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
