hy00nc commented on a change in pull request #223: [NEMO-388] Off-heap memory
management (reuse ByteBuffer)
URL: https://github.com/apache/incubator-nemo/pull/223#discussion_r307094944
##########
File path:
runtime/executor/src/main/java/org/apache/nemo/runtime/executor/data/partition/SerializedPartition.java
##########
@@ -226,4 +236,11 @@ public int getLength() throws IOException {
public boolean isOffheap() {
return offheap;
}
+
+ /**
+ * Releases the off-heap memory that this SerializedPartition holds.
+ */
+ public void release() {
+ bytesOutputStream.release();
Review comment:
Actually, `release()` operates on the **data**, although it seems like it
works on the stream. The stream here uses a list of ByteBuffers wrapped as
MemoryChunk, which is the data itself. The `dataList` in SerializedPartition
shares the list of ByteBuffers of the output stream, so if this is called upon
commit(), it will return the 'uncleared' MemoryChunks back to the pool, causing
correctness problem. Maybe I could add some comments on this?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
