lordcheng10 commented on issue #16782: URL: https://github.com/apache/pulsar/issues/16782#issuecomment-1221974845
> Do you have some numbers about experiments ? Before accepting the proposal we must be sure that this algorithm works on only on the paper but also on some real benchmark/test env Test steps： 1.create Topic: ``` sh bin/pulsar-admin tenants create test-tenant1 sh bin/pulsar-admin namespaces create test-tenant1/test-namespace1 sh bin/pulsar-admin topics create-partitioned-topic test-tenant1/test-namespace1/test1 -p 20 sh bin/pulsar-admin topics create-partitioned-topic test-tenant1/test-namespace1/test2 -p 20 sh bin/pulsar-admin topics create-partitioned-topic test-tenant1/test-namespace1/test3 -p 20 sh bin/pulsar-admin topics create-partitioned-topic test-tenant1/test-namespace1/test4 -p 20 sh bin/pulsar-admin topics create-partitioned-topic test-tenant1/test-namespace1/test5 -p 20 sh bin/pulsar-admin topics create-partitioned-topic test-tenant1/test-namespace1/test6 -p 20 sh bin/pulsar-admin topics create-partitioned-topic test-tenant1/test-namespace1/test7 -p 20 sh bin/pulsar-admin topics create-partitioned-topic test-tenant1/test-namespace1/test8 -p 20 sh bin/pulsar-admin topics create-partitioned-topic test-tenant1/test-namespace1/test9 -p 20 sh bin/pulsar-admin topics create-partitioned-topic test-tenant1/test-namespace1/test10 -p 20 sh bin/pulsar-admin topics create-partitioned-topic test-tenant1/test-namespace1/test11 -p 20 sh bin/pulsar-admin topics create-partitioned-topic test-tenant1/test-namespace1/test12 -p 20 sh bin/pulsar-admin topics create-partitioned-topic persistent://test-tenant1/test-namespace1/test695 -p 1 ``` 2.generate data: ``` nohup sh bin/pulsar-perf produce test-tenant1/test-namespace1/test1 -threads 1 -n 1 -r 1000 > log1 & nohup sh bin/pulsar-perf produce test-tenant1/test-namespace1/test2 -threads 1 -n 1 -r 1000 > log2 & nohup sh bin/pulsar-perf produce test-tenant1/test-namespace1/test3 -threads 1 -n 1 -r 1000 > log3 & nohup sh bin/pulsar-perf produce test-tenant1/test-namespace1/test4 -threads 1 -n 1 -r 1000 > log4 & nohup sh bin/pulsar-perf produce test-tenant1/test-namespace1/test5 -threads 1 -n 1 -r 1000 > log5 & nohup sh bin/pulsar-perf produce test-tenant1/test-namespace1/test6 -threads 1 -n 1 -r 1000 > log6 & nohup sh bin/pulsar-perf produce test-tenant1/test-namespace1/test7 -threads 1 -n 1 -r 1000 > log7 & nohup sh bin/pulsar-perf produce test-tenant1/test-namespace1/test8 -threads 1 -n 1 -r 1000 > log8 & nohup sh bin/pulsar-perf produce test-tenant1/test-namespace1/test9 -threads 1 -n 1 -r 1000 > log9 & nohup sh bin/pulsar-perf produce test-tenant1/test-namespace1/test10 -threads 1 -n 1 -r 1000 > log10 & nohup sh bin/pulsar-perf produce test-tenant1/test-namespace1/test11 -threads 1 -n 1 -r 1000 > log11 & nohup sh bin/pulsar-perf produce test-tenant1/test-namespace1/test12 -threads 1 -n 1 -r 1000 > log12 & nohup sh bin/pulsar-perf produce persistent://test-tenant1/test-namespace1/test695 -threads 1 -n 11 -r 1010 > log695 & ``` 3.the configuration is as follows: #defaultNamespaceBundleSplitAlgorithm=topic_count_equally_divide defaultNamespaceBundleSplitAlgorithm=flow_or_qps_equally_divide loadBalancerNamespaceBundleMaxMsgRate=1010 flowOrQpsDifferenceThresholdPercentage=10 exposeBundlesMetricsInPrometheus=true The comparison effect is as follows: 1.Using the topic_count_equally_divide algorithm, pulsar_bundle_msg_rate_in is monitored as follows: The number of bundles after splitting is 21, and the pulsar_bundle_msg_rate_in of a single bundle is: 49 msg/s ~ 1010 msg/s：  2.Using the new algorithm flow_or_qps_equally_divide the effect is as follows: The number of bundles after splitting is 13, and the pulsar_bundle_msg_rate_in of a single bundle is: 997 msg/s ~ 1010 msg/s： <img width="1282" alt="image" src="https://user-images.githubusercontent.com/19296967/185864580-198b92fb-ef0b-4390-9bfc-56a03a1b19f0.png";> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
![https://user-images.githubusercontent.com/19296967/185864580-198b92fb-ef0b-4390-9bfc-56a03a1b19f0.png"](https://user-images.githubusercontent.com/19296967/185864580-198b92fb-ef0b-4390-9bfc-56a03a1b19f0.png"){kind=link}