[
https://issues.apache.org/jira/browse/CASSANDRA-11303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15194055#comment-15194055
]
Paulo Motta commented on CASSANDRA-11303:
-----------------------------------------
Thanks for the patch [~skonno]. Overall your approach looks good, you need to
make a few adjustments:
* As you already noticed, limiting the inbound throughput will be a bit
trickier because the data consumption will be performed internally by
{{SSTableMultiWriter.append}}. One idea is to wrap the {{LZFInputStream}} into
a new {{ThrottledInputStream}} (extending {{FilterInputStream}}) that receives
a {{RateLimiter}} object and transparently acquires the permit before each
{{read*}} operation.
* The same static {{RateLimiter}} object is being shared between inbound and
outgoing streams on {{StreamRateLimiter}}, you'll need to have 4 static
{{RateLimiter}} objects: {{inboundLimiter, outboundLimiter,
inboundInterDCLimiter, outboundInterDCLimiter}}.
* As {{StreamRateLimiter}} will become more complex, I think it can move to its
own file instead of being part of {{StreamManager}}
* Add property and description to {{cassandra.yaml}}
* Add similar support to {{CompressedStreamReader}}
After you have your initial version ready, if you could raise a test cluster
(check it out [ccm|https://github.com/pcmanus/ccm]) and check if the new
property is working as expected that would be great.
> New inbound throughput parameters for streaming
> -----------------------------------------------
>
> Key: CASSANDRA-11303
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11303
> Project: Cassandra
> Issue Type: New Feature
> Components: Configuration
> Reporter: Satoshi Konno
> Priority: Minor
> Attachments: cassandra_inbound_stream.diff
>
>
> Hi,
> To specify stream throughputs of a node more clearly, I would like to add the
> following new inbound parameters like existing outbound parameters in the
> cassandra.yaml.
> - stream_throughput_inbound_megabits_per_sec
> - inter_dc_stream_throughput_outbound_megabits_per_sec
> We use only the existing outbound parameters now, but it is difficult to
> control the total throughputs of a node. In our production network, some
> critical alerts occurs when a node exceed the specified total throughput
> which is the sum of the input and output throughputs.
> In our operation of Cassandra, the alerts occurs during the bootstrap or
> repair processing when a new node is added. In the worst case, we have to
> stop the operation of the exceed node.
> I have attached the patch under consideration. I would like to add a new
> limiter class, StreamInboundRateLimiter, and use the limiter class in
> StreamDeserializer class. I use Row::dataSize( ）to get the input throughput
> in StreamDeserializer::newPartition(), but I am not sure whether the
> dataSize() returns the correct data size.
> Can someone please tell me how to do it ?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
