[
https://issues.apache.org/jira/browse/CASSANDRA-579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12792026#action_12792026
]
Stu Hood edited comment on CASSANDRA-579 at 12/17/09 5:53 PM:
--------------------------------------------------------------
> 1. use SSTR.getPosition to find start and end ranges to transfer...
I don't think this works: you still need to perform the compaction first, which
means you still need to dump to disk.
EDIT: Unless you are suggesting streaming each SSTable individually?
> 2. from the data file, compute index + BF files on the destination node
This seems like the best approach.
was (Author: stuhood):
> 1. use SSTR.getPosition to find start and end ranges to transfer...
I don't think this works: you still need to perform the compaction first, which
means you still need to dump to disk.
> 2. from the data file, compute index + BF files on the destination node
This seems like the best approach.
> Add support to io.Streaming API for sending Streams
> ---------------------------------------------------
>
> Key: CASSANDRA-579
> URL: https://issues.apache.org/jira/browse/CASSANDRA-579
> Project: Cassandra
> Issue Type: Improvement
> Reporter: Stu Hood
> Fix For: 0.9
>
>
> The io.Streaming API currently requires a file on disk to stream, which means
> that bootstrap and repairs need to perform an anti-compaction that writes a
> bunch of data to disk, only to have it be deleted after the streaming has
> finished.
> Ideally, the Streaming API should allow for streaming from an InputStream (or
> any other class we think we need to design to make the streaming as efficient
> as possible). That way, anti-compaction for repair/bootstrap does not perform
> any writing: it simply streams the relevant portion of the file to the
> neighbor.
> Additionally, this opens up interesting possibilities, such as providing the
> Streaming API as a (Java only?) client API. One use case would be for a
> Hadoop OutputFormat: rather than writing BinaryMemtables, the OutputFormat
> could literally write an SSTable to the stream. This might require better
> integration with gossip, to ensure that you aren't writing to the completely
> wrong node.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
