hrsakai commented on issue #3535: Deduplication causes a lot of Full GCs URL: https://github.com/apache/pulsar/issues/3535#issuecomment-463480871 We kept producing messages for 4 hours and memory usage of broker became as follow: ``` Deduplication disabled : 50G Deduplication enabled: 99G(Reached 99G and full GC occured) ``` If deduplication is disabled, memory usage remained at 50G all the time. On the other hand, if deduplication is enabled, memory usage gradually increased from 50G to 99G. ■memory settings -Xms96g -Xmx96g -XX:MaxDirectMemorySize=96g ■deduplication settings brokerDeduplicationMaxNumberOfProducers=10000 brokerDeduplicationEntriesInterval=1000 brokerDeduplicationProducerInactivityTimeoutMinutes=360 ■the number of partition 8 (We used a partitioned topic for test) ■Rate 20,000msg/s(2,500msg/s per internal topic) ■Message size 200byte ■Execution time 4hours
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
