lordcheng10 opened a new pull request #3061:
URL: https://github.com/apache/bookkeeper/pull/3061


   ### Motivation
   We found that the pulsar broker frequently appeared Full Gc, and found by 
dumping the heap memory: org.apache.pulsar.broker.service.Consumer#pendingAcks 
occupies 9G of memory. The pendingAcks variable is defined as follows:
   
![image](https://user-images.githubusercontent.com/19296967/154415440-79c221fd-41cb-4e9c-b847-d20fdfc5a6ae.png)
   
   The heap memory usage is as follows:
   
![image](https://user-images.githubusercontent.com/19296967/154415596-8eac70f3-184d-4bc2-a3a1-449fe22dfd47.png)
   
   I found that ConcurrentLongLongPairHashMap only supports expansion, not 
shrinkage, and most of the memory is not used and wasted.
   
   Of course, this structure is used not only in pulsar, but also in the read 
and write cache of bookkeeper, which will also lead to a lot of memory waste:
   
![image](https://user-images.githubusercontent.com/19296967/154415958-fdd8adb3-a4fb-47a7-948d-069a3f02593a.png)
   
   
   ### Changes
   
   When removing, judge whether to shrink. If the current use size is less than 
resizeThresholdBelow, then shrinking is required.
    if (size < resizeThresholdBelow) {
                       try {
                           // shrink the hashmap
                           rehash(capacity / 2);
                       } finally {
                           unlockWrite(stamp);
                       }
                   } else {
                       unlockWrite(stamp);
                   }


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


Reply via email to