[
https://issues.apache.org/jira/browse/LUCENE-5006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Simon Willnauer updated LUCENE-5006:
------------------------------------
Attachment: LUCENE-5006.patch
here is a first/rough cut with some nocommits but it shows the idea. This patch
removes all dependencies to DocumentsWriter and IndexWriter from DWPT. I
factored everything out and made DWPT basically a non-reuseable class. It
initializes itself entirely in the ctor including the segment name etc.
At the same time DWPTThreadPool doesn't initialize DWPT anymore it only handles
ThreadStates and pooling and the actual DWPT creation is done on the
ThreadPools consumer level. This makes the pool much simpler as well.
DocumentsWriter doesn't communicate with the IW directly. It only creates
certain events that will be processed by the IW once the DW action is done.
This works since all the Events are kind of atomic operations which can be
executed once we exit DW. For instance if we flush a segment the DW doesn't
push it up to the IW but once DW returns IW proceses the event an pulls the new
flushed segment in. The same is true for pending merges etc.
This simplifies the locking a lot here since we basically can't deadlock
anymore from DW or DWPT since we don't even have a reference to IW anymore. The
only remaining thing is when we create a new DWPT we need to call into IW to
get a new seg. name but it's a start.
> Simplify / understand IndexWriter/DocumentsWriter synchronization
> -----------------------------------------------------------------
>
> Key: LUCENE-5006
> URL: https://issues.apache.org/jira/browse/LUCENE-5006
> Project: Lucene - Core
> Issue Type: Bug
> Reporter: Michael McCandless
> Assignee: Simon Willnauer
> Attachments: LUCENE-5006.patch
>
>
> The concurrency in IW/DW/BD is terrifying: there are many locks involved, not
> just intrinsic locks but IW also has fullFlushLock, commitLock, and there are
> no clear rules about lock order to avoid deadlocks like LUCENE-5002.
> We have to somehow simplify this, and define the allowed concurrent behavior
> eg when an app calls deleteAll while other threads are indexing.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
