[
https://issues.apache.org/jira/browse/CASSANDRA-16?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12879339#action_12879339
]
Jonathan Ellis commented on CASSANDRA-16:
-----------------------------------------
1. updated comments in new 03
2. added patch 07 to address the low hanging fruit here (see below)
3. fixed by removing rowSize in favor of dataSize in new 04
4. added class javadoc to ACR, PCR, and LCR in new 01 and 04
I experimented with making more invasive changes to SSTII (making SSTS
implement iterable instead of iterator, and making it iterate over SSTableRow
objects that could build a SSTII or just read an entire CF, so we could move
getColumnFamilyWithColumns out of SSTII) but this made things worse overall;
the requirements for row scans vs compaction are just different enough to make
things ugly.
> Memory efficient compactions
> -----------------------------
>
> Key: CASSANDRA-16
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Environment: All
> Reporter: Sandeep Tata
> Assignee: Jonathan Ellis
> Fix For: 0.7
>
> Attachments: 0001-introduce-AbstractCompactedRow-PrecompactedRow.txt,
> 0002-make-single-pass-over-columns-for-indexing.txt,
> 0003-r-m-object-count-abomination.-fix-BF-serialization-to-.txt,
> 0004-add-LazilyCompactedRow.txt, 0005-make-row-size-64-bits.txt,
> 0006-make-row-size-at-which-to-drop-to-incremental-compacti.txt,
> 0007-reduce-seeks-in-SSTableIdentityIterator-methods.txt
>
>
> The basic idea is to allow rows to get large enough that they don't have to
> fit in memory entirely, but can easily fit on a disk. The compaction
> algorithm today de-serializes the entire row in memory before writing out the
> compacted SSTable (see ColumnFamilyStore.doCompaction() and associated
> methods).
> The requirement is to have a compaction method with a lower memory
> requirement so we can support rows larger than available main memory. To
> re-use the old FB example, if we stored a user's inbox in a row, we'd want
> the inbox to grow bigger than memory so long as it fit on disk.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
