[
https://issues.apache.org/jira/browse/LUCENE-6209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14297742#comment-14297742
]
Yonik Seeley commented on LUCENE-6209:
--------------------------------------
Good info to have.
Would it make sense to generally expose this info (i.e. some API to retrieve
the duration of the last N stalls?)
> IndexWriter should confess when it stalls flushes
> -------------------------------------------------
>
> Key: LUCENE-6209
> URL: https://issues.apache.org/jira/browse/LUCENE-6209
> Project: Lucene - Core
> Issue Type: Improvement
> Reporter: Michael McCandless
> Assignee: Michael McCandless
> Fix For: Trunk, 5.1
>
> Attachments: LUCENE-6209.patch
>
>
> You tell IW how much RAM it's allowed to use to hold recently indexed
> documents before they must be written to disk.
> IW is willing to use up to 2X that amount for in-progress flushes.
> If the in-progress flushes go over that limit, then IW will stall them,
> hijacking indexing threads and having them wait until the in-progress flushes
> are below 2X indexing buffer size again.
> This is important back-pressure e.g. if you are indexing on a machine with
> many cores but slowish IO.
> Often when I profile an indexing heavy use case, even on fast IO (SSD) boxes,
> I see the methods associated with this back-pressure taking unexpected time
> ... yet IW never logs when it stalls/unstalls flushing. I think it should.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
