rickyma commented on code in PR #1531:
URL:
https://github.com/apache/incubator-uniffle/pull/1531#discussion_r1494558928
##########
server/src/main/java/org/apache/uniffle/server/NettyDirectMemoryTracker.java:
##########
@@ -68,6 +74,9 @@ public void start() {
ShuffleServerMetrics.gaugeUsedDirectMemorySize.set(usedDirectMemory);
ShuffleServerMetrics.gaugeAllocatedDirectMemorySize.set(allocatedDirectMemory);
ShuffleServerMetrics.gaugePinnedDirectMemorySize.set(pinnedDirectMemory);
+ if (nettyServerEnabled) {
+ shuffleBufferManager.setUsedMemory(pinnedDirectMemory);
Review Comment:
You can never accurately obtain the `usedMemory` calculated through business
code.
The first reason is that you cannot estimate this size, due to a lot reasons
mentioned before, like network fluctuations.
The second reason is that `PooledByteBufAllocator` may reuse direct memory
through caching.
That means even if you calculate the size directly through the `ByteBuf`
received in the method `channedRead` on server side, the `usedMemory` you count
may still be larger than the memory managed by `PooledByteBufAllocator`.
Moreover, NettyUtils.getNettyBufferAllocator().pinnedDirectMemory() is very
performance-consuming, so it is periodically obtained.
So, it is meant to be not determined anyways. And we don't need a determined
`usedMemory` here. That's why I use a `calculateEstimatedMemoryAllocationSize`
method to calculate the size.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
