pnowojski commented on a change in pull request #12120:
URL: https://github.com/apache/flink/pull/12120#discussion_r426121598
##########
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/io/disk/FileBasedBufferIterator.java
##########
@@ -85,6 +90,6 @@ private int read(byte[] buffer) {
@Override
public void close() throws Exception {
- closeAll(stream, file::release);
+ closeAll(stream, file::release, () ->
buffersToClose.forEach(Buffer::recycleBuffer));
Review comment:
Why are we recycling buffers at the end, on close and not on the fly? It
can mean GBs of extra (unnecessary) memory usage if we keep all unconsumed
buffers from different channels. The ownership of the buffers returned from the
iterator should belong to the caller of `next()` method.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
