Anonymitaet commented on a change in pull request #12180:
URL: https://github.com/apache/pulsar/pull/12180#discussion_r716156123
##########
File path:
pulsar-broker/src/main/java/org/apache/pulsar/broker/loadbalance/impl/ThresholdShedder.java
##########
@@ -35,6 +35,18 @@
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
+/**
+ * Load shedding strategy that will unload any broker that exceeds the average
resource utilization of all brokers by a
Review comment:
```suggestion
* Load shedding strategy that unloads any broker that exceeds the average
resource utilization of all brokers by a
```
##########
File path:
pulsar-broker/src/main/java/org/apache/pulsar/broker/loadbalance/impl/OverloadShedder.java
##########
@@ -35,11 +35,14 @@
/**
* Load shedding strategy which will attempt to shed exactly one bundle on
brokers which are overloaded, that is, whose
- * maximum system resource usage exceeds
loadBalancerBrokerOverloadedThresholdPercentage. A bundle will be recommended
- * for unloading off that broker if and only if the following conditions hold:
The broker has at least two bundles
- * assigned and the broker has at least one bundle that has not been unloaded
recently according to
+ * maximum system resource usage exceeds
loadBalancerBrokerOverloadedThresholdPercentage. To see which resources are
+ * considered when determining the maximum system resource, see {@link
LocalBrokerData#getMaxResourceUsage()}. A bundle
+ * will be recommended for unloading off that broker if and only if the
following conditions hold: The broker has at
+ * least two bundles assigned and the broker has at least one bundle that has
not been unloaded recently according to
* LoadBalancerSheddingGracePeriodMinutes. The unloaded bundle will be the
most expensive bundle in terms of message
- * rate that has not been recently unloaded.
+ * rate that has not been recently unloaded. Note that this strategy does not
take into account "underloaded" brokers
+ * when determining which bundles to unload. If you are looking for a strategy
that will spread load evenly across
Review comment:
```suggestion
* when determining which bundles to unload. If you are looking for a
strategy that spreads load evenly across
```
##########
File path:
pulsar-broker/src/main/java/org/apache/pulsar/broker/loadbalance/impl/ThresholdShedder.java
##########
@@ -35,6 +35,18 @@
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
+/**
+ * Load shedding strategy that will unload any broker that exceeds the average
resource utilization of all brokers by a
+ * configured threshold. As a consequence, this strategy tends to distribute
load among all brokers. It does this by
+ * first computing the average resource usage per broker for the whole
cluster. The resource usage for each broker is
+ * calculated using the following method: {@link
LocalBrokerData#getMaxResourceUsageWithWeight)}. The weights for each
+ * resource are configurable. Historical observations are included in the
running average based on the broker's
+ * setting for loadBalancerHistoryResourcePercentage. Once the average
resource usage is calculated, a broker's
+ * current/historical usage is compared to the average broker usage. If a
broker's usage is greater than the average
+ * usage per broker plus the loadBalancerBrokerThresholdShedderPercentage,
this load shedder will propose removing
Review comment:
```suggestion
* usage per broker plus the loadBalancerBrokerThresholdShedderPercentage,
this load shedder proposes removing
```
##########
File path:
pulsar-broker/src/main/java/org/apache/pulsar/broker/loadbalance/impl/ThresholdShedder.java
##########
@@ -35,6 +35,18 @@
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
+/**
+ * Load shedding strategy that will unload any broker that exceeds the average
resource utilization of all brokers by a
+ * configured threshold. As a consequence, this strategy tends to distribute
load among all brokers. It does this by
+ * first computing the average resource usage per broker for the whole
cluster. The resource usage for each broker is
+ * calculated using the following method: {@link
LocalBrokerData#getMaxResourceUsageWithWeight)}. The weights for each
+ * resource are configurable. Historical observations are included in the
running average based on the broker's
+ * setting for loadBalancerHistoryResourcePercentage. Once the average
resource usage is calculated, a broker's
+ * current/historical usage is compared to the average broker usage. If a
broker's usage is greater than the average
+ * usage per broker plus the loadBalancerBrokerThresholdShedderPercentage,
this load shedder will propose removing
+ * enough bundles to bring the unloaded broker 5% below the current average
broker usage. Note that recently
+ * unloaded bundles will not be unloaded again.
Review comment:
```suggestion
* unloaded bundles are not unloaded again.
```
##########
File path:
pulsar-broker/src/main/java/org/apache/pulsar/broker/loadbalance/impl/OverloadShedder.java
##########
@@ -35,11 +35,14 @@
/**
* Load shedding strategy which will attempt to shed exactly one bundle on
brokers which are overloaded, that is, whose
- * maximum system resource usage exceeds
loadBalancerBrokerOverloadedThresholdPercentage. A bundle will be recommended
- * for unloading off that broker if and only if the following conditions hold:
The broker has at least two bundles
- * assigned and the broker has at least one bundle that has not been unloaded
recently according to
+ * maximum system resource usage exceeds
loadBalancerBrokerOverloadedThresholdPercentage. To see which resources are
+ * considered when determining the maximum system resource, see {@link
LocalBrokerData#getMaxResourceUsage()}. A bundle
+ * will be recommended for unloading off that broker if and only if the
following conditions hold: The broker has at
Review comment:
```suggestion
* is recommended for unloading off that broker if and only if the following
conditions hold: The broker has at
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
