eolivelli commented on code in PR #3600:
URL: https://github.com/apache/celeborn/pull/3600#discussion_r2799576984
##########
worker/src/main/scala/org/apache/celeborn/service/deploy/worker/storage/FlushTask.scala:
##########
@@ -41,7 +41,7 @@ abstract private[worker] class FlushTask(
copyBytes: Array[Byte],
length: Int): Array[Byte] = {
if (copyBytes != null && copyBytes.length >= length) {
- buffer.readBytes(copyBytes, 0, length)
+ buffer.getBytes(buffer.readerIndex(), copyBytes, 0, length)
Review Comment:
the main difference is here:
https://github.com/apache/celeborn/blob/e168fd54906b23011a9ada8d27b564f25c6c9920/worker/src/main/scala/org/apache/celeborn/service/deploy/worker/storage/Flusher.scala#L120-L123
when it "returns" the buffer we use the "readableBytes" to tell
MemoryManager that the buffer has been returned
without this fix the variable `diskBufferCounter` in MemoryManager keeps
increasing and never goes down
https://github.com/apache/celeborn/blob/77a8921648c77a6fd073a09e07aa6f41a98153c1/worker/src/main/java/org/apache/celeborn/service/deploy/worker/memory/MemoryManager.java#L72
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
