[
https://issues.apache.org/jira/browse/LUCENE-1717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12724074#action_12724074
]
Simon Willnauer commented on LUCENE-1717:
-----------------------------------------
Regarding buffered delete's RAM usage, accounting an exact number is quite
difficult in this case as there are many strings involved (Terms with field and
value) . BufferedDeletes#terms stores <Term, Num> and BufferedDeletes#queries
stores <Query, Num> in both cases the value part is easy to account while
especially for query the memory consumption is hard to guess similarly the
amount of memory a Term takes.
On the other hand I would like to have a notion of memory consumption os
BufferedDeletes but the IndexWriters#setRAMBufferSizeMB javaDoc clearly says
that this does not include the memory used by buffered deletes. I would rather
tend to leave it as it is and make it clear in javadoc / wiki that
setMaxBufferedDeleteTerms is the way to go if you run into memory problems.
Feels quite ambiguous to estimate the memory of buffered deletes.
bq. I think is a good default once we fix the accounting in IndexWriter to
properly account for buffered delete's RAM usage.
is there already an issue to fix the RAM usage?
> IndexWriter does not properly account for the RAM consumed by pending deletes
> -----------------------------------------------------------------------------
>
> Key: LUCENE-1717
> URL: https://issues.apache.org/jira/browse/LUCENE-1717
> Project: Lucene - Java
> Issue Type: Bug
> Affects Versions: 2.4, 2.4.1
> Reporter: Michael McCandless
> Fix For: 2.9
>
> Attachments: BufferedDeletes_beautification.patch
>
>
> IndexWriter, with autoCommit false, is able to carry buffered deletes for
> quite some time before materializing them to docIDs (thus freeing up RAM
> used).
> It's only on triggering a merge (or, commit/close) that the deletes are
> materialized and the RAM is freed.
> I expect this in practice is a smallish amount of RAM, but we should still
> fix it.
> I don't have a patch yet so if someone wants to grab this, feel free!!
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
