[
https://issues.apache.org/jira/browse/CASSANDRA-5342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marcus Eriksson updated CASSANDRA-5342:
---------------------------------------
Attachment: 0001-CASSANDRA-5342-wip.patch
removes ancestors from SSTableMetadata and instead makes deserialize(..) return
a Pair<SSTM, Set<Integer>> so that the caller can decide if the ancestors are
needed. This allows us to keep SSTM as immutable as possible.
this forces us to re-deserialize the metadata when trying to figure out
ancestors during compaction
opted not to mutate the ancestors on-disk since it makes my skin crawl
> ancestors are not cleared in SSTableMetadata after compactions are done and
> old SSTables are removed
> ----------------------------------------------------------------------------------------------------
>
> Key: CASSANDRA-5342
> URL: https://issues.apache.org/jira/browse/CASSANDRA-5342
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.1.3
> Reporter: Wei Zhu
> Assignee: Marcus Eriksson
> Fix For: 1.2.7
>
> Attachments: 0001-CASSANDRA-5342-wip.patch, Screen Shot 2013-03-13 at
> 12.05.08 PM.png
>
>
> We are using LCS and have total of 38000 SSTables for one CF. During LCS,
> there could be over a thousand SSTable involved. All those SSTable IDs are
> stored in ancestors field of SSTableMetatdata for the new table. In our case,
> it consumes more than 1G of heap memory for those field. Put it in
> perspective, the ancestors consume 2 - 3 times more memory than bloomfilter
> (fp = 0.1 by default) in LCS.
> We should remove those ancestors from SSTableMetadata after the compaction is
> finished and the old SSTable is removed. It might be a big deal for Sized
> Compaction since there are small number of SSTable involved. But it consumes
> a lot of memory for LCS.
> At least, we shouldn't load those ancestors to the memory during startup if
> the files are removed.
> I would love to contribute and provide patch. Please let me know how to
> start.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
