lordcheng10 commented on pull request #12471: URL: https://github.com/apache/pulsar/pull/12471#issuecomment-955736427
> I wonder if it would make more sense to implement a new `LoadSheddingStrategy` to achieve your goals instead of updating the `ThesholdShedder` class. This class is only meant to unload bundles from a broker with above average (plus some threshold) usage. > > Based on your PR's description, you'd like the `ThresholdShedder` algorithm to take under utilized brokers into special consideration. As @hangc0276 pointed out, it is not straightforward to consider these brokers because the only mechanism we have available in this class is to "unload" a bundle from a broker. Your proposed changes require the unloading of bundles from brokers that are not above the average (plus some threshold) usage, which seems like a new strategy to me. > > Note that because we're using a simple average, clusters with many brokers might have a few under utilized brokers without any one broker being too much above the average utilization. An alternative strategy could give underutilized brokers more weight. > > > The calculated avgUsage=(80*100+10)/101=79 at this time, if the threshold is set to the default value of 10, then any broker will satisfy currentUsage <avgUsage + threshold, so that the balancing operation will not be triggered, so some load Low machines cannot be fully utilized. > > Based on your example, an alternative solution is to configure a smaller threshold. Decreasing the threshold will lead to better distribution, but the trade off is that it will also lead to more frequent unloading, which has costs too. > > If we do decide to update this class's strategy, we will need to document the new details in the Javadoc as well as add tests to ensure that the algorithm works correctly. My modified balance strategy is not to unload from the Brokers below the average value, but to calculate the total load that the Brokers below the average value can receive, so as to calculate the ratio of unloading from the Brokers above the average value. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
