hangc0276 commented on pull request #9261: URL: https://github.com/apache/pulsar/pull/9261#issuecomment-766307076
For BatchReceive, I add batchReceive policy for PerformanceConsumer #9295 , and test as follow 1. create topic with 30 partitions (persistent://test/test/test_batch_v1) 1. run perf producer at high performance 2. run consumer with batch receive policy and start from latest to ensure read message from cache, in order to avoid read bookie entry disk perf command as follow: 1. run perf producer ``` bin/pulsar-perf produce -u pulsar://xxx:6650 -i 10 -r 500000 -threads 5 -ioThreads 3 persistent://test/test/test_batch_v1 ``` 2. run consumer using pulsar-master lib and pulsar-master-patch lib ``` bin/pulsar-perf consume -u pulsar://xxx:6650 -i 10 -sp Latest -time 240 -s batch-perf-v1 -q 10000 -bb 1048576 -bm 100 -bt 100 persistent://test/test/test_batch_v1 ``` The result as follow | version | msgOut (msg/s) | bytesOut (Mbit/s) | | --- | --- | --- | | pulsar-master | 110081.956 | 860.015 | | pulsar-master-patch | 116298.462 | 908.582 | The consume performance improved 5.6% The `pulsar-master-patch` version just remove the lock for batchReceive module. I will move the lock back for batchReceive module. @codelipenghui @315157973 ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
