[
https://issues.apache.org/jira/browse/CASSANDRA-9318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15382114#comment-15382114
]
Sergio Bossa commented on CASSANDRA-9318:
-----------------------------------------
I've pushed a new update based on the most recent discussions, more
specifically:
* I've implemented (and locally tested) rate limiting based on either the
fastest or slowest replica; rate limiting on the average rate required
computing such rate for all replicas in the whole cluster each RPC timeout
interval (2 secs by default), which I deemed to be not worth it.
* I've removed the low ratio configuration: if we aim to keep the cluster as
"responsive" as possible, throwing exceptions or prematurely hinting doesn't
make sense.
* {{SP.sendToHintedEndpoints}} is largely unchanged: given we only rate limit
by a single (fastest/slowest) replica, sorting the endpoints by back-pressure
doesn't make sense either.
Answering to [~Stefania]'s latest comments:
bq. The proposal doesn't address: "The API should provide for reporting load to
clients so they can do real load balancing across coordinators and not just
round-robin."
This is misleading: the implemented back-pressure mechanism is cross-replica,
so load balancing to different coordinators would be useless.
bq. we can add metrics or another mechanism later on in follow up tickets, as
well as consider implementing the memory-threshold strategy.
Metrics can be added in this ticket once we settle on the core algorithm, but
then yes any reporting mechanism to clients should be probably dealt with
separately as it would probably involve changes to the native protocol (and I'm
not sure what's the usual procedure in such case).
Regarding the memory-threshold strategy, I would like to stress once again the
fact it's a coordinator-only back-pressure mechanism which wouldn't directly
benefit replicas; also, [~aweisberg] showed in his tests that such strategy
isn't even needed given the local hints back-pressure/overflowing mechanisms
already implemented. So my vote goes against it, but if you/others really want
it, I'm perfectly fine implementing it, but it will have to be in this ticket,
as it requires some more changes to the back-pressure API.
> Bound the number of in-flight requests at the coordinator
> ---------------------------------------------------------
>
> Key: CASSANDRA-9318
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9318
> Project: Cassandra
> Issue Type: Improvement
> Components: Local Write-Read Paths, Streaming and Messaging
> Reporter: Ariel Weisberg
> Assignee: Sergio Bossa
> Attachments: 9318-3.0-nits-trailing-spaces.patch, backpressure.png,
> limit.btm, no_backpressure.png
>
>
> It's possible to somewhat bound the amount of load accepted into the cluster
> by bounding the number of in-flight requests and request bytes.
> An implementation might do something like track the number of outstanding
> bytes and requests and if it reaches a high watermark disable read on client
> connections until it goes back below some low watermark.
> Need to make sure that disabling read on the client connection won't
> introduce other issues.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
