[
https://issues.apache.org/jira/browse/LUCENE-6209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael McCandless updated LUCENE-6209:
---------------------------------------
Attachment: LUCENE-6209.patch
Simple patch; it adds messages like this into into the infoStream:
{noformat}
DW 0 [2015-01-29 15:41:22.289; Thread-3]: now stalling flushes
...
DW 0 [2015-01-29 15:41:22.328; Thread-3]: done stalling flushes for 38.150627 ms
{noformat}
Correction: it does in fact tell you now when stalling is happening, but this
patch adds additional logging telling you how long it was stalled for.
> IndexWriter should confess when it stalls flushes
> -------------------------------------------------
>
> Key: LUCENE-6209
> URL: https://issues.apache.org/jira/browse/LUCENE-6209
> Project: Lucene - Core
> Issue Type: Improvement
> Reporter: Michael McCandless
> Assignee: Michael McCandless
> Fix For: Trunk, 5.1
>
> Attachments: LUCENE-6209.patch
>
>
> You tell IW how much RAM it's allowed to use to hold recently indexed
> documents before they must be written to disk.
> IW is willing to use up to 2X that amount for in-progress flushes.
> If the in-progress flushes go over that limit, then IW will stall them,
> hijacking indexing threads and having them wait until the in-progress flushes
> are below 2X indexing buffer size again.
> This is important back-pressure e.g. if you are indexing on a machine with
> many cores but slowish IO.
> Often when I profile an indexing heavy use case, even on fast IO (SSD) boxes,
> I see the methods associated with this back-pressure taking unexpected time
> ... yet IW never logs when it stalls/unstalls flushing. I think it should.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
