rdhabalia commented on a change in pull request #634: PIP-3 : Introduce 
message-dispatch rate limiting
URL: https://github.com/apache/incubator-pulsar/pull/634#discussion_r132031203
 
 

 ##########
 File path: 
pulsar-broker/src/main/java/org/apache/pulsar/broker/service/persistent/PersistentTopic.java
 ##########
 @@ -1366,5 +1388,80 @@ public void markBatchMessagePublished() {
         this.hasBatchMessagePublished = true;
     }
 
+    public RateLimiter getDispatchRateLimiter() {
+        return dispatchRateLimiter;
+    }
+    
+    public DispatchRateType getDispatchRateType() {
+        return dispatchRateType;
+    }
+
+    public boolean tryDispatchPermit(long permits) {
+        return permits <= 0 || dispatchRateLimiter == null
+        // acquiring permits must be < configured msg-rate;
+                || dispatchRateLimiter.tryAcquire(permits);
+    }
+    
+    public boolean hasMessageDispatchPermit() {
+        return dispatchRateLimiter == null || 
dispatchRateLimiter.getAvailablePermits() > 0;
+    }
+
+    private void updateDispatchRate(LocalZooKeeperCacheService localZKCache) {
+        final String path = joinPath(LOCAL_POLICIES_ROOT, 
DestinationName.get(this.topic).getNamespace());
+        try {
+            Optional<LocalPolicies> policies = 
localZKCache.policiesCache().getAsync(path).get(cacheTimeOutInSec,
+                    SECONDS);
+            if (policies.isPresent() && policies.get().dispatchRate != null
+                    && policies.get().dispatchRate.topicDispatchRate > 0) {
+                updateDispatchRate(policies.get().dispatchRate);
+                log.info("[{}] configured message-dispatch rate configured at 
policy {}}", this.topic,
+                        policies.get().dispatchRate);
+                return;
+            }
+        } catch (Exception e) {
+            log.warn("Failed to get message-rate for {}", this.topic, e);
+        }
+        DispatchRate dispatchRate = 
brokerService.pulsar().getConfiguration().getDispatchRatePerTopicInMsg() > 0
 
 Review comment:
   > Actually, that could just be done inside DispatchRate class
   
   Actually, using admin-api, we always define 
`DispatchRateType=msgRate/byteRate` and throttling-value in `DispatchRate` 
class.
   However, for dynamic configuration we can't store complex-type so, created 
two variables `dispatchRatePerTopicInMsg` and `dispatchRatePerTopicInByte`, 
therefore, we need this logic when we have value set for both configuration.
   
   > Ideally, if both limits are set, both of them should be applied.
   
   This will require 2 `RateLimiter` objects for every topic which we use 
across all the subscriptions. Do you think considering both limits would be 
useful?
 
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

Reply via email to