siddharthteotia commented on a change in pull request #4472: Use hit counter to
track max QPS per minute for broker
URL: https://github.com/apache/incubator-pinot/pull/4472#discussion_r308470001
##########
File path:
pinot-broker/src/main/java/org/apache/pinot/broker/queryquota/HitCounter.java
##########
@@ -77,11 +90,154 @@ public int getHitCount() {
int getHitCount(long timestamp) {
long numTimeUnits = timestamp / _timeBucketWidthMs;
int count = 0;
- for (int i = 0; i < BUCKET_COUNT; i++) {
- if (numTimeUnits - _bucketStartTime.get(i) < BUCKET_COUNT) {
+ for (int i = 0; i < _bucketCount; i++) {
+ if (numTimeUnits - _bucketStartTime.get(i) < _bucketCount) {
count += _bucketHitCount.get(i);
}
}
return count;
}
+
+ /*
+ * Explanation of algorithm to keep track of
+ * max QPS per minute:
+ *
+ * Hit counter is configured with fixed number buckets
+ * and each bucket covers a fixed time window to
+ * cover an overall range of 60secs.
+ *
+ * Each bucket stores two values -- start time
Review comment:
I am not quite sure what that implies so we should discuss it. Let's take an
example:
Start time of bucket b1 is 100 and bucket b2 is 101. Now if there are 6
buckets, then 106 and 107 will also map to b1 and b2 respectively.
When there are only 2 hits recorded for 100 and 101 start times the latter
can probably be derived from former. But let's say 3rd hit comes after a window
of 1 min and let say start time is 107 then b2's start time will be set to 107
and this is something that can't be derived from b1's start time of 100
(although b1's hit count has now become irrelelevant as it is not out of 60sec
window)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
