[
https://issues.apache.org/jira/browse/CASSANDRA-5342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13701217#comment-13701217
]
Jonathan Ellis commented on CASSANDRA-5342:
-------------------------------------------
LGTM.
Nit: since we only ever use either the metadata, or the anscestors, it may be
cleaner to split deserializeAncestors into a separate method. OTOH that might
make it easier to make mistakes in compatibility. I'm fine either way.
> ancestors are not cleared in SSTableMetadata after compactions are done and
> old SSTables are removed
> ----------------------------------------------------------------------------------------------------
>
> Key: CASSANDRA-5342
> URL: https://issues.apache.org/jira/browse/CASSANDRA-5342
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.1.3
> Reporter: Wei Zhu
> Assignee: Marcus Eriksson
> Fix For: 1.2.7
>
> Attachments: 0001-CASSANDRA-5342-wip.patch,
> 0001-CASSANDRA-5342-wip-v2.patch, Screen Shot 2013-03-13 at 12.05.08 PM.png
>
>
> We are using LCS and have total of 38000 SSTables for one CF. During LCS,
> there could be over a thousand SSTable involved. All those SSTable IDs are
> stored in ancestors field of SSTableMetatdata for the new table. In our case,
> it consumes more than 1G of heap memory for those field. Put it in
> perspective, the ancestors consume 2 - 3 times more memory than bloomfilter
> (fp = 0.1 by default) in LCS.
> We should remove those ancestors from SSTableMetadata after the compaction is
> finished and the old SSTable is removed. It might be a big deal for Sized
> Compaction since there are small number of SSTable involved. But it consumes
> a lot of memory for LCS.
> At least, we shouldn't load those ancestors to the memory during startup if
> the files are removed.
> I would love to contribute and provide patch. Please let me know how to
> start.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
