[
https://issues.apache.org/jira/browse/CASSANDRA-847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12842152#action_12842152
]
Jonathan Ellis edited comment on CASSANDRA-847 at 3/6/10 2:03 AM:
------------------------------------------------------------------
> I've always considered the Iterator interface to be super-weak for IO. There
> is no support for exception handling, and hasNext() is dangerous (when you
> begin chaining iterators that are returning lazy objects like IteratingRow)
> because it usually involves seeks.
Statically checked exceptions are a misfeature anyway, so that's a non-issue.
hasNext won't seek any more than a clunkier manual interface. either way you
need to (a) design your format so you don't have to do random i/o and (b)
buffer to avoid seeks when switching between different objects, and that is
what we do.
the old iterator-based compaction for instance never does more seeks than
another interface would given the same buffer sizes.
was (Author: jbellis):
> I've always considered the Iterator interface to be super-weak for IO.
There is no support for exception handling, and hasNext() is dangerous (when
you begin chaining iterators that are returning lazy objects like IteratingRow)
because it usually involves seeks.
Typed exceptions are a misfeature anyway, so that's a non-issue.
hasNext won't seek any more than a clunkier manual interface. either way you
need to (a) design your format so you don't have to do random i/o and (b)
buffer to avoid seeks when switching between different objects, and that is
what we do.
the old iterator-based compaction for instance never does more seeks than
another interface would given the same buffer sizes.
> Make the reading half of compactions memory-efficient
> -----------------------------------------------------
>
> Key: CASSANDRA-847
> URL: https://issues.apache.org/jira/browse/CASSANDRA-847
> Project: Cassandra
> Issue Type: Improvement
> Reporter: Stu Hood
> Priority: Critical
> Fix For: 0.7
>
> Attachments:
> 0001-Add-structures-that-were-important-to-the-SSTableSca.patch,
> 0002-Implement-most-of-the-new-SSTableScanner-interface.patch,
> 0003-Rename-RowIndexedReader-specific-test.patch,
> 0004-Improve-Scanner-tests-and-separate-SuperCF-handling-.patch,
> 0005-Add-Scanner-interface-and-a-Filtered-implementation-.patch,
> 0006-Add-support-for-compaction-of-super-CFs-and-some-tes.patch
>
>
> This issue is the next on the road to finally fixing CASSANDRA-16. To make
> compactions memory efficient, we have to be able to perform the compaction
> process on the smallest possible chunks that might intersect and contend
> one-another, meaning that we need a better abstraction for reading from
> SSTables.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
