rickyma commented on code in PR #1531:
URL:
https://github.com/apache/incubator-uniffle/pull/1531#discussion_r1492719640
##########
server/src/main/java/org/apache/uniffle/server/buffer/ShuffleBuffer.java:
##########
@@ -47,6 +47,8 @@ public class ShuffleBuffer {
private final long capacity;
private long size;
+ // for Netty mode
+ private long estimatedSize;
Review Comment:
Because we use the accurate real-time used direct memory
`pinnedDirectMemory` to determine whether to do the pre-allocation(or flush) or
not. If we use `size` to calculate `usedMemory`, `usedMemory` will gradually
deviate from `pinnedDirectMemory` over time(In fact, they will deviate more and
more, with an increasing divergence.). This will lead to inaccuracies when
calling the `pickFlushedShuffle` method and when the coordinator allocates
shuffle servers, as they both continue to use `usedMemory` as the basis for
judgment.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
