[GitHub] [pulsar] AnonHxy commented on pull request #15033: [improvement][client-java] Avoid too large memory preallocation for batch message.

[GitBox]

AnonHxy commented on PR #15033:
URL: https://github.com/apache/pulsar/pull/15033#issuecomment-1257728382

   >  Is there a significant throughput difference between these two?
   
   OK.  @Jason918 
   1. First I setup up a standalone cluster using apache-pulsar-2.10.1
   2. Then started a producer like below to publish messages, starting with 
command `~/houxiaoyu/jdk-17.0.4/bin/java -Xms2000m -Xmx2000m 
-XX:MaxDirectMemorySize=4000m -jar pulsar-buffer.jar`:
   ```
     public static void main(String[] args) throws Exception {
           String topic = "hxy-test-mem";
           byte[] smallMsg = new byte[64];
           PulsarClient client = PulsarClient.builder()
                   .serviceUrl("pulsar://127.0.0.1:6650")
                   .memoryLimit(2000, SizeUnit.MEGA_BYTES)  // memory limit 
2000M
                   .build();
           Producer<byte[]> producer = client.newProducer()
                   .topic(topic)
                   .enableBatching(true)
                   .create();
           for (int i = 0; i < Integer.MAX_VALUE; ++i) {
               producer.sendAsync(smallMsg);
           }
       }
   ```
   3.  I have compared the producer stats three times, and all of them came to 
the same conclusion that throughput will be descended when using 
`compositeBuffer`.  Here is the details logs:
      *  first time test:
     
         buffer：
   <img width="1919" alt="image" 
src="https://user-images.githubusercontent.com/10233437/192234832-02ed6234-1d1d-4b30-9daf-812895577cb5.png";>
        compositeBuffer：
   <img width="1919" alt="image" 
src="https://user-images.githubusercontent.com/10233437/192235247-75d5a8bc-c7d6-4127-9e21-a546f36ec29b.png";>
   
     * second time test
     
       buffer:
   <img width="1919" alt="image" 
src="https://user-images.githubusercontent.com/10233437/192236074-930351a5-9798-4b24-ae54-93b8280b03a1.png";>
       compositeBuffer：
   <img width="1918" alt="image" 
src="https://user-images.githubusercontent.com/10233437/192236240-279ec3dd-f4d3-4ae9-96a2-38623cd2c7ce.png";>
   
      * third time test
      
       buffer:
   <img width="1919" alt="image" 
src="https://user-images.githubusercontent.com/10233437/192236536-4f8c25dd-54cb-4be2-a3e6-2324221dd873.png";>
   
      compositeBuffer：   
   <img width="1919" alt="image" 
src="https://user-images.githubusercontent.com/10233437/192236617-ff4a96e5-fb7a-4fec-8ec4-9b228da2a43f.png";>
   
   
   
   
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org