thetumbled commented on code in PR #21011: URL: https://github.com/apache/pulsar/pull/21011#discussion_r1301234939
########## pip/pip-294.md: ########## @@ -0,0 +1,66 @@ +# Background knowledge + +There are mainly two `LoadManager` implementation in Pulsar broker: `ExtensibleLoadManager` and `ModularLoadManagerImpl`. `ModularLoadManagerImpl` is the default load manager, and `ExtensibleLoadManager` is a new load manager which is proposed after 3.0.0 version. + +## ModularLoadManagerImpl +`ModularLoadManagerImpl` rely on zk to store and synchronize metadata about load, which pose greate pressure on zk, threatening the stability of system. Every broker will upload its `LocalBrokerData` to zk, and leader broker will retrieve all `LocalBrokerData` from zk, generate all `BundleData` from each `LocalBrokerData`, and update all `BundleData` to zk. + +## ExtensibleLoadManager +`ExtensibleLoadManager` depends on system topics and table views for load balance metadata store and replication. Though not using zk to store and synchronize metadata about load, it is still necessary to control the number of bundles that need to be updated, for which there is a `loadBalancerMaxNumberOfBundlesInBundleLoadReport` configuration in `ExtensibleLoadManager` that select the top k bundles. Review Comment: `loadBalancerMaxNumberOfBundlesInBundleLoadReport` is the configuration for topK mechanism, which do not work in `ModularLoadManagerImpl`. TopK mechanism is implemented only in new LoadManager. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
