[
https://issues.apache.org/jira/browse/CASSANDRA-579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12791771#action_12791771
]
Chris Goffinet commented on CASSANDRA-579:
------------------------------------------
We see about 16MB/s for compaction on a 4 disk RAID0 configuration. Moving this
directly to network, so we can be network bound would be much better. But the
thing is, compaction is running slower than the max throughput on network,
which is roughly 80MB/s on our systems.
> Add support to io.Streaming API for sending Streams
> ---------------------------------------------------
>
> Key: CASSANDRA-579
> URL: https://issues.apache.org/jira/browse/CASSANDRA-579
> Project: Cassandra
> Issue Type: Improvement
> Reporter: Stu Hood
> Fix For: 0.9
>
>
> The io.Streaming API currently requires a file on disk to stream, which means
> that bootstrap and repairs need to perform an anti-compaction that writes a
> bunch of data to disk, only to have it be deleted after the streaming has
> finished.
> Ideally, the Streaming API should allow for streaming from an InputStream (or
> any other class we think we need to design to make the streaming as efficient
> as possible). That way, anti-compaction for repair/bootstrap does not perform
> any writing: it simply streams the relevant portion of the file to the
> neighbor.
> Additionally, this opens up interesting possibilities, such as providing the
> Streaming API as a (Java only?) client API. One use case would be for a
> Hadoop OutputFormat: rather than writing BinaryMemtables, the OutputFormat
> could literally write an SSTable to the stream. This might require better
> integration with gossip, to ensure that you aren't writing to the completely
> wrong node.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
