[
https://issues.apache.org/jira/browse/LUCENE-6209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14298698#comment-14298698
]
ASF subversion and git services commented on LUCENE-6209:
---------------------------------------------------------
Commit 1656029 from [~mikemccand] in branch 'dev/trunk'
[ https://svn.apache.org/r1656029 ]
LUCENE-6209: IndexWriter now logs (to infoStream) how much time flushing
threads were stalled because of > 2X IW's RAM buffer in flush backlog
> IndexWriter should confess when it stalls flushes
> -------------------------------------------------
>
> Key: LUCENE-6209
> URL: https://issues.apache.org/jira/browse/LUCENE-6209
> Project: Lucene - Core
> Issue Type: Improvement
> Reporter: Michael McCandless
> Assignee: Michael McCandless
> Fix For: Trunk, 5.1
>
> Attachments: LUCENE-6209.patch
>
>
> You tell IW how much RAM it's allowed to use to hold recently indexed
> documents before they must be written to disk.
> IW is willing to use up to 2X that amount for in-progress flushes.
> If the in-progress flushes go over that limit, then IW will stall them,
> hijacking indexing threads and having them wait until the in-progress flushes
> are below 2X indexing buffer size again.
> This is important back-pressure e.g. if you are indexing on a machine with
> many cores but slowish IO.
> Often when I profile an indexing heavy use case, even on fast IO (SSD) boxes,
> I see the methods associated with this back-pressure taking unexpected time
> ... yet IW never logs when it stalls/unstalls flushing. I think it should.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
