scwhittle commented on code in PR #32961:
URL: https://github.com/apache/beam/pull/32961#discussion_r1825772064
##########
sdks/java/core/src/main/java/org/apache/beam/sdk/fn/stream/DataStreams.java:
##########
@@ -202,6 +202,12 @@ public WeightedList<T>
decodeFromChunkBoundaryToChunkBoundary() {
T next = next();
rvals.add(next);
}
+ // We don't support seeking backwards so release the memory of the last
Review Comment:
No this happens whenever decoding an element ends up completing consuming
the current bytestring. Since we are communicating entire elements over the
fnapi that occurs for every block.
(In fact it seems like a bug otherwise since we are looping over
decodeFromChunkBoundaryToChunkBoundary above this and this method discards the
currentStream on line 194; if there were bytes in it they would be lost.)
previousStream would only not equal currentStream in the case where the
decoder needed to move to the next bytestring to finish parsing the next
element. If that did happen, we would no longer be holding onto currentStream
and it would be eligible for collection anyway.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
