[
https://issues.apache.org/jira/browse/CASSANDRA-3668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13920122#comment-13920122
]
Joshua McKenzie commented on CASSANDRA-3668:
--------------------------------------------
With streaming being overhauled in 2.0 it looks like I'll need to revisit the
core implementation on this.
Got the following from Yuki:
"In streaming 2.0, transferring/receiving files to one destination is managed
by StreamSession object.
To parallelize streaming, I think the easiest way is to make multiple
StreamSession per host, instead of one per host that what we do right now.
So given set of SSTable files in bulk loader, divide those up to groups equal
to number of parallel streaming, and assign each group to multiple
StreamSessions of the same destination.
One thing to be careful in this approach is, you need to add one more id to
streaming messages in order to distinguish messages to the same destination.
This also changes serialization format of the protocol so need to update
streaming protocol version defined in StreamMessage."
> Parallel streaming for sstableloader
> ------------------------------------
>
> Key: CASSANDRA-3668
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3668
> Project: Cassandra
> Issue Type: Improvement
> Components: API
> Reporter: Manish Zope
> Assignee: Joshua McKenzie
> Priority: Minor
> Labels: streaming
> Fix For: 2.1 beta2
>
> Attachments: 3668-1.1-v2.txt, 3668-1.1.txt,
> 3688-reply_before_closing_writer.txt, sstable-loader performance.txt
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> One of my colleague had reported the bug regarding the degraded performance
> of the sstable generator and sstable loader.
> ISSUE :- https://issues.apache.org/jira/browse/CASSANDRA-3589
> As stated in above issue generator performance is rectified but performance
> of the sstableloader is still an issue.
> 3589 is marked as duplicate of 3618.Both issues shows resolved status.But the
> problem with sstableloader still exists.
> So opening other issue so that sstbleloader problem should not go unnoticed.
> FYI : We have tested the generator part with the patch given in 3589.Its
> Working fine.
> Please let us know if you guys require further inputs from our side.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
