It’s very great that Apache Kafka get a feature of setting quota since 0.9.
https://kafka.apache.org/documentation/#design_quotas
But it only provide a method of shell bash ：
bin/kafka-configs.sh --zookeeper localhost:2181 --alter --add-config
'producer_byte_rate=1024,consumer_byte_rate=2048,request_percentage=200'--entity-typeclients
--entity-name clientA
Updated config forentity: client-id'clientA'.
And now, I don't want to login kafka serve. And I want apply the quota feature
in my Java Application.
I don't find any API in the kafka-clients maven dependency.
And I try to set zookeeper data directly. Sadly, it does not work.
So, does any one has these this similar condition?
./bin/kafka-producer-perf-test.sh --topic test --num-records 300 --record-size
100 --throughput 15 --producer-props
bootstrap.servers=hadoop01:9093,hadoop02:9093,hadoop03:9093 client.id=clientE
before set zkï¼
77 records sent, 15.2 records/sec (0.00 MB/sec), 5.6 ms avg latency, 231.0 max
latency.
76 records sent, 15.1 records/sec (0.00 MB/sec), 1.6 ms avg latency, 3.0 max
latency.
76 records sent, 15.1 records/sec (0.00 MB/sec), 1.6 ms avg latency, 2.0 max
latency.
300 records sent, 15.006753 records/sec (0.00 MB/sec), 2.59 ms avg latency,
231.00 ms max latency, 2 ms 50th, 2 ms 95th, 19 ms 99th, 231 ms 99.9th
create /config/clients/clientE
{"version":1,"config":{"request_percentage":"200","producer_byte_rate":"1024","consumer_byte_rate":"2048"}}
after set zkï¼
77 records sent, 15.2 records/sec (0.00 MB/sec), 5.1 ms avg latency, 218.0 max
latency.
76 records sent, 15.1 records/sec (0.00 MB/sec), 1.5 ms avg latency, 2.0 max
latency.
76 records sent, 15.1 records/sec (0.00 MB/sec), 1.4 ms avg latency, 2.0 max
latency.
300 records sent, 15.010507 records/sec (0.00 MB/sec), 2.38 ms avg latency,
218.00 ms max latency, 1 ms 50th, 2 ms 95th, 14 ms 99th, 218 ms 99.9th.
./bin/kafka-configs.sh --zookeeper hadoop01:2181,hadoop02:2181,hadoop03:2181
--alter --add-config 'producer_byte_rate=1024,consumer_byte_rate=2048'
--entity-type clients --entity-name clientE
set quota with shellï¼
55 records sent, 10.1 records/sec (0.00 MB/sec), 81.9 ms avg latency, 1883.0
max latency.
7 records sent, 1.1 records/sec (0.00 MB/sec), 5119.6 ms avg latency, 7799.0
max latency.
81 records sent, 11.8 records/sec (0.00 MB/sec), 7185.1 ms avg latency, 9297.0
max latency.
42 records sent, 4.1 records/sec (0.00 MB/sec), 8531.4 ms avg latency, 16766.0
max latency.
get /config/clients/clientE
{"version":1,"config":{"request_percentage":"200","producer_byte_rate":"1024","consumer_byte_rate":"2048"}}
set /config/clients/clientE
{"version":1,"config":{"request_percentage":"200","producer_byte_rate":"10240","consumer_byte_rate":"2048"}}
after set zk to add quotaï¼
55 records sent, 10.1 records/sec (0.00 MB/sec), 81.5 ms avg latency, 1883.0
max latency.
7 records sent, 1.1 records/sec (0.00 MB/sec), 5118.7 ms avg latency, 7798.0
max latency.
81 records sent, 11.8 records/sec (0.00 MB/sec), 7113.0 ms avg latency, 9299.0
max latency.
42 records sent, 4.2 records/sec (0.00 MB/sec), 8527.0 ms avg latency, 16551.0
max latency.
300 records sent, 9.273284 records/sec (0.00 MB/sec), 8639.50 ms avg latency,
16551.00 ms max latency, 8089 ms 50th, 15645 ms 95th, 16418 ms 99th, 16551 ms
99.9th.
