ASF GitHub Bot commented on KAFKA-4178:

GitHub user benstopford opened a pull request:


    KAFKA-4178: Windows in Rate Calculation

    ReplicationQuotas included a new Rate class which changed the way the 
window is calculated. @junrao asked that we look to consolidate this. On 
balance I believe there is a good case for both, so this PR pulls out two 
policies for calculating the window, and attempts to explain what they are and 
why we might need them. 
    They are:
    **Elapsed**: Set the window to the difference between the oldest and newest 
measurement time. Handle NaN. _(Replication Quotas use this)_
    **Fixed**: Fix the window to the full duration (10s by default). _(Client 
Quotas use this)_
    Replication Quotas uses **Elapsed** as **Fixed** provides underestimates 
during the first window. Underestimates are best avoided as they create a load 
spike when replication starts. So it's preferable to overestimate the rate, and 
hence increase the throttled rate slowly. Elapsed is also significantly easier 
to test. 
    I'm totally sure why Client Quotas were changed to use **Fixed**. I know 
there was a NaN issue, but should really be tangential (it is handled in both 
policies). However it seems sensible to slowly throttle on a client's 
connection down to the desired rate when you initialise or change quotas. 
    So I think there is a requirement for both types of rate, and hence I've 
included both in this PR. **Elapsed** suits throttled replication, where the 
general use case is to initiate some rebalance and immediately apply a 
conservative throttle. **Fixed** suits client quotas where we want to gently 
throttle a client down to the desired rate over a period of seconds. 

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/benstopford/kafka KAFKA-4178

Alternatively you can review and apply these changes as the patch at:


To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #1893


> Replication Throttling: Consolidate Rate Classes
> ------------------------------------------------
>                 Key: KAFKA-4178
>                 URL: https://issues.apache.org/jira/browse/KAFKA-4178
>             Project: Kafka
>          Issue Type: Improvement
>          Components: replication
>    Affects Versions:
>            Reporter: Ben Stopford
> Replication throttling is using a different implementation of Rate to client 
> throttling (Rate & SimpleRate). These should be consolidated so both use the 
> same approach. 

This message was sent by Atlassian JIRA

Reply via email to