[
https://issues.apache.org/jira/browse/LUCENE-2283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12838975#action_12838975
]
Michael McCandless commented on LUCENE-2283:
--------------------------------------------
Patch looks great -- thanks Tim!
A few fixes:
* I think you should pass false to allocator.getByteBlock in
PerDocBuffer.newBuffer. We don't want this allocator to track allocations
because this storage used is transient (reset per doc). We only pass true for
things like the terms hash, that keeps allocating RAM until it's time to flush.
* Can you also make the same changes to TermVectorsTermsWriter? I think the
same hazard applies to it. It should just use the same pool.
* Can you add a CHANGES entry?
Thanks!
> Possible Memory Leak in StoredFieldsWriter
> ------------------------------------------
>
> Key: LUCENE-2283
> URL: https://issues.apache.org/jira/browse/LUCENE-2283
> Project: Lucene - Java
> Issue Type: Bug
> Affects Versions: 2.4.1
> Reporter: Tim Smith
> Assignee: Michael McCandless
> Fix For: 3.1
>
> Attachments: LUCENE-2283.patch
>
>
> StoredFieldsWriter creates a pool of PerDoc instances
> this pool will grow but never be reclaimed by any mechanism
> furthermore, each PerDoc instance contains a RAMFile.
> this RAMFile will also never be truncated (and will only ever grow) (as far
> as i can tell)
> When feeding documents with large number of stored fields (or one large
> dominating stored field) this can result in memory being consumed in the
> RAMFile but never reclaimed. Eventually, each pooled PerDoc could grow very
> large, even if large documents are rare.
> Seems like there should be some attempt to reclaim memory from the PerDoc[]
> instance pool (or otherwise limit the size of RAMFiles that are cached) etc
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-dev-h...@lucene.apache.org
