[GitHub] [pulsar] HQebupt opened a new pull request, #16059: [improve][broker] add switch for enable/disable distribute bundles evenly in LoadManager

[GitBox]

HQebupt opened a new pull request, #16059:
URL: https://github.com/apache/pulsar/pull/16059

   ### Motivation
   When we use `ModualLoadManager` as `LoadManager` in the pulsar cluster, and 
the load balancer shedding strategy is `ThresholdShedder`, we found that the 
unloaded bundles might be loaded by another broker which has resource usage 
above average load, and make the new broker be overloaded again, which 
frequently causes unloading bundles.
   ```
   
loadManagerClassName=org.apache.pulsar.broker.loadbalance.impl.ModularLoadManagerImpl
   
loadBalancerLoadSheddingStrategy=org.apache.pulsar.broker.loadbalance.impl.ThresholdShedder
   loadBalancerBrokerThresholdShedderPercentage=10
   ```
   We found that it tooks more than 10 hours and 400+ times to doing load 
shedding  in the cluster. And the cluster continue to unload bundles when some 
of topics are lots of traffic  in a short time. Below is the unload metric.
   <img width="866" alt="image" 
src="https://user-images.githubusercontent.com/4970972/173602319-95e7b7a3-e2c9-45c3-a9af-9cd7e88ea227.png";>
   
   The key reason is that some brokers with lower load  but more bundles can 
not be candidate due to distributing bundles evenly in LoadManager by force. 
   <img width="1186" alt="image" 
src="https://user-images.githubusercontent.com/4970972/173603516-2efefae4-d22b-445c-9414-6744d22e0323.png";>
   
   It could be much better to disable distribute bundles evenly in 
`LoadManager`, which can select the broker from those having resource usage 
below average load, so it can prevent the least loaded broker from quickly 
becoming heavily loaded.
   
   Therefore, it recommend that enable distribute bundles evenly among all 
brokers by customers according to user scenarios .
   
   After disable distribute bundles evenly in `LoadManager`, the brokers with 
lower load but more bunldes can be candidate. It reduce the unload times and 
the cluster is stable with even load on each broker as follows.
   <img width="1661" alt="image" 
src="https://user-images.githubusercontent.com/4970972/173606832-92eeac3f-89ff-4291-b90d-2a6de7ad3b84.png";>
   
   
   ### Modifications
   - Add a field property `loadBalancerDistributeBundlesEvenlyEnabled` for Load 
Balancer in `ServiceConfiguration`, keep it to be true as default.
   - Add a switch to enable distribute bundles evenly in `ModularLoadManager`
   
   ### Verifying this change
   
   - [x]  Make sure that the change passes the CI checks.
   
   This change is a trivial rework / code cleanup without any test coverage.
   ### Does this pull request potentially affect one of the following parts:
   
   If `yes` was chosen, please highlight the changes
   
   - Dependencies (does it add or upgrade a dependency): (no)
   - The public API: (no)
   - The schema: (no)
   - The default values of configurations: (no)
   - The wire protocol: (no)
   - The rest endpoints: (no)
   - The admin cli options: (no)
   - Anything that affects deployment: (no)
   
   ### Documentation
   Check the box below and label this PR (if you have committer privilege).
   
   Need to update docs? 
   - [x] `no-need-doc` 
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org