lordcheng10 commented on issue #16782: URL: https://github.com/apache/pulsar/issues/16782#issuecomment-1222292922
> > Do you have some numbers about experiments ? Before accepting the proposal we must be sure that this algorithm works on only on the paper but also on some real benchmark/test env > > > > ### Test steps： > > 1.create Topic: > > > > ``` > > sh bin/pulsar-admin tenants create test-tenant1 > > sh bin/pulsar-admin namespaces create test-tenant1/test-namespace1 > > sh bin/pulsar-admin topics create-partitioned-topic test-tenant1/test-namespace1/test1 -p 20 > > sh bin/pulsar-admin topics create-partitioned-topic test-tenant1/test-namespace1/test2 -p 20 > > sh bin/pulsar-admin topics create-partitioned-topic test-tenant1/test-namespace1/test3 -p 20 > > sh bin/pulsar-admin topics create-partitioned-topic test-tenant1/test-namespace1/test4 -p 20 > > sh bin/pulsar-admin topics create-partitioned-topic test-tenant1/test-namespace1/test5 -p 20 > > sh bin/pulsar-admin topics create-partitioned-topic test-tenant1/test-namespace1/test6 -p 20 > > sh bin/pulsar-admin topics create-partitioned-topic test-tenant1/test-namespace1/test7 -p 20 > > sh bin/pulsar-admin topics create-partitioned-topic test-tenant1/test-namespace1/test8 -p 20 > > sh bin/pulsar-admin topics create-partitioned-topic test-tenant1/test-namespace1/test9 -p 20 > > sh bin/pulsar-admin topics create-partitioned-topic test-tenant1/test-namespace1/test10 -p 20 > > sh bin/pulsar-admin topics create-partitioned-topic test-tenant1/test-namespace1/test11 -p 20 > > sh bin/pulsar-admin topics create-partitioned-topic test-tenant1/test-namespace1/test12 -p 20 > > sh bin/pulsar-admin topics create-partitioned-topic persistent://test-tenant1/test-namespace1/test695 -p 1 > > ``` > > > > 2.generate data: > > > > ``` > > nohup sh bin/pulsar-perf produce test-tenant1/test-namespace1/test1 -threads 1 -n 1 -r 1000 > log1 & > > nohup sh bin/pulsar-perf produce test-tenant1/test-namespace1/test2 -threads 1 -n 1 -r 1000 > log2 & > > nohup sh bin/pulsar-perf produce test-tenant1/test-namespace1/test3 -threads 1 -n 1 -r 1000 > log3 & > > nohup sh bin/pulsar-perf produce test-tenant1/test-namespace1/test4 -threads 1 -n 1 -r 1000 > log4 & > > nohup sh bin/pulsar-perf produce test-tenant1/test-namespace1/test5 -threads 1 -n 1 -r 1000 > log5 & > > nohup sh bin/pulsar-perf produce test-tenant1/test-namespace1/test6 -threads 1 -n 1 -r 1000 > log6 & > > nohup sh bin/pulsar-perf produce test-tenant1/test-namespace1/test7 -threads 1 -n 1 -r 1000 > log7 & > > nohup sh bin/pulsar-perf produce test-tenant1/test-namespace1/test8 -threads 1 -n 1 -r 1000 > log8 & > > nohup sh bin/pulsar-perf produce test-tenant1/test-namespace1/test9 -threads 1 -n 1 -r 1000 > log9 & > > nohup sh bin/pulsar-perf produce test-tenant1/test-namespace1/test10 -threads 1 -n 1 -r 1000 > log10 & > > nohup sh bin/pulsar-perf produce test-tenant1/test-namespace1/test11 -threads 1 -n 1 -r 1000 > log11 & > > nohup sh bin/pulsar-perf produce test-tenant1/test-namespace1/test12 -threads 1 -n 1 -r 1000 > log12 & > > nohup sh bin/pulsar-perf produce persistent://test-tenant1/test-namespace1/test695 -threads 1 -n 11 -r 1010 > log695 & > > ``` > > > > 3.the configuration is as follows: > > ``` > > #defaultNamespaceBundleSplitAlgorithm=topic_count_equally_divide > > defaultNamespaceBundleSplitAlgorithm=flow_or_qps_equally_divide > > loadBalancerNamespaceBundleMaxMsgRate=1010 > > flowOrQpsDifferenceThresholdPercentage=10 > > exposeBundlesMetricsInPrometheus=true > > defaultNumberOfNamespaceBundles=1 > > ``` > > > > ### The comparison effect is as follows: > > **1.Using the topic_count_equally_divide algorithm:** > > > > The number of bundles after splitting is **21**, and the **pulsar_bundle_msg_rate_in** of a single bundle is: **49 msg/s ~ 1010 msg/s**： > > > >  > > > > > > > > **2.Using the new algorithm flow_or_qps_equally_divide:** > > The number of bundles after splitting is **13**, and the **pulsar_bundle_msg_rate_in** of a single bundle is: **997 msg/s ~ 1010 msg/s**： > > <img width="1282" alt="image" src="https://user-images.githubusercontent.com/19296967/185864580-198b92fb-ef0b-4390-9bfc-56a03a1b19f0.png";> > > > > > > @eolivelli @Technoboy- @codelipenghui PTAL,thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
![https://user-images.githubusercontent.com/19296967/185864580-198b92fb-ef0b-4390-9bfc-56a03a1b19f0.png"](https://user-images.githubusercontent.com/19296967/185864580-198b92fb-ef0b-4390-9bfc-56a03a1b19f0.png"){kind=link}