[
https://issues.apache.org/jira/browse/CASSANDRA-847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12842130#action_12842130
]
Stu Hood commented on CASSANDRA-847:
------------------------------------
> "one huge BF" is still a bad idea
I wasn't attempting to make a decision on the BF issue with this code: the
ColumnKey BF methods are remnants from 674 which I will happily remove. I think
we agreed at some point that having a bloom filter per SSTable block was going
to be a better approach, and to leave the global bloom filter as-is.
> the Scanner api seems like a step back from IteratingRow to me.
> self-contained iterators are good
I definitely agree re encapsulation, but I've always considered the Iterator
interface to be super-weak for IO. There is no support for exception handling,
and hasNext() is dangerous (when you begin chaining iterators that are
returning lazy objects like IteratingRow) because it usually involves seeks.
> scanners can return IR
You'll note that that method is deprecated: IteratingRow is sticking around as
a shim in this case, because I didn't want to spend too much time porting
SSTable(Im|Ex)port to the Slice API quite yet. The same goes for other
@Deprecated methods on the SSTableScanner and SSTableReader base classes: I
hope that we can remove them when the new file format is added.
> telling bad sign: CompactionIterator is 2x as long as it used to be.
This is mostly because I replaced ReducingIterator with a custom merge-sort
which may or may not be slower anway. Additionally, we're doing a lot of work
to collect slices into CFs that we can remove once the writing half of
compactions is memory efficient.
> Make the reading half of compactions memory-efficient
> -----------------------------------------------------
>
> Key: CASSANDRA-847
> URL: https://issues.apache.org/jira/browse/CASSANDRA-847
> Project: Cassandra
> Issue Type: Improvement
> Reporter: Stu Hood
> Priority: Critical
> Fix For: 0.7
>
> Attachments:
> 0001-Add-structures-that-were-important-to-the-SSTableSca.patch,
> 0002-Implement-most-of-the-new-SSTableScanner-interface.patch,
> 0003-Rename-RowIndexedReader-specific-test.patch,
> 0004-Improve-Scanner-tests-and-separate-SuperCF-handling-.patch,
> 0005-Add-Scanner-interface-and-a-Filtered-implementation-.patch,
> 0006-Add-support-for-compaction-of-super-CFs-and-some-tes.patch
>
>
> This issue is the next on the road to finally fixing CASSANDRA-16. To make
> compactions memory efficient, we have to be able to perform the compaction
> process on the smallest possible chunks that might intersect and contend
> one-another, meaning that we need a better abstraction for reading from
> SSTables.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
