wenbingshen opened a new issue #12169:
URL: https://github.com/apache/pulsar/issues/12169


   **Describe the bug**
   A clear and concise description of what the bug is.
   
   Pulsar and bookkeeper version:
   pulsar-2.8.0 and pulsar-2.8.0 built-in bookkeeper
   
   In order to figure out the reason for the OOM of the pulsar broker's direct 
memory, I tested different scenarios and got some different results.
   
   After analyzing the pulsar broker heap dump, a large number of PendingAddOp 
instances have not been restored or destroyed.
   
   As shown in the figure below, I suspect that a large number of entry 
requests written to bookie have not received all the WQ responses, which makes 
PendingAddOp unable to be recycled or destroyed.
   
   
![image](https://user-images.githubusercontent.com/35599757/134630405-64fec4d3-2714-4c8a-968b-bf65e6bd75bd.png)
   
   Therefore, I use maxMessagePublishBufferSizeInMB to limit the traffic 
handled by the broker according to https://github.com/apache/pulsar/pull/7406 
and https://github.com/apache/pulsar/pull/6178.
   
   But next is my test results:
   1. The broker is configured with maxMessagePublishBufferSizeInMB=512, EW 
A=3:3:2, OOM still occurs after the pressure test
   2. The broker configures maxMessagePublishBufferSizeInMB=512, and tests EW 
A=3:3:3, 3:2:2, and 2:2:2 respectively. After the pressure test, the direct 
memory is normal
   3. The broker configures maxMessagePublishBufferSizeInMB=2048, test EW 
A=3:3:3 and 3:2:2, after the pressure test, the direct memory is normal
   4. The broker configuration keeps maxMessagePublishBufferSizeInMB as the 
default value, the default is 1/2 of the maximum allocated off-heap memory 
(8/2=4GB in the test), test EW A=3:3:3 and 3:2:2, pressure test The off-heap 
memory is normal
   5. The broker configures maxMessagePublishBufferSizeInMB=-1, closes current 
limiting measures, tests EW A=3:3:3 and 3:2:2, the memory is normal after the 
pressure test
   6. The broker configures maxMessagePublishBufferSizeInMB=-1, closes current 
limiting measures, tests EW A=3:3:2, OOM occurs after the pressure test
   
   The next questions also are related to #9562 
   
   My question is, whether maxMessagePublishBufferSizeInMB is configured or 
not, 
   as long as AQ=WQ, direct memory is normal, 
   as long as AQ<WQ, direct memory will appear OOM, 
   then how does maxMessagePublishBufferSizeInMB work?
   
   
   
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


Reply via email to