nandini12396 commented on PR #21102: URL: https://github.com/apache/kafka/pull/21102#issuecomment-3641389891
Hi @showuon thanks for your review! Pooling heap buffers would reduce allocation frequency but doesn't solve the core issue. Heap buffers still trigger humongous allocation and get scanned by GC every cycle. Direct buffers move the data off-heap entirely into native memory where GC doesn't see it. We also get zero-copy I/O since the data is already in native memory for socket writes. To share some test results, with heap buffers, GC ran every 100ms with 1.3GB retained after each collection; with direct buffers, GC dropped to every 30-40 seconds with only 325MB retained. I will remove the metrics changes from this PR so we can focus on the buffer pool implementation. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
