sunhaibotb commented on a change in pull request #8471: [FLINK-12529][runtime]
Release record-deserializer buffers timely to improve the efficiency of heap
usage on taskmanager
URL: https://github.com/apache/flink/pull/8471#discussion_r303365251
##########
File path:
flink-streaming-java/src/main/java/org/apache/flink/streaming/runtime/io/StreamTwoInputProcessor.java
##########
@@ -307,21 +307,35 @@ private void processBufferOrEvent(BufferOrEvent
bufferOrEvent) throws IOExceptio
if (event.getClass() != EndOfPartitionEvent.class) {
throw new IOException("Unexpected event: " +
event);
}
+
+ // release the record deserializer immediately,
+ // which is very valuable in case of bounded stream
+ releaseDeserializer(bufferOrEvent.getChannelIndex());
}
}
public void cleanup() throws IOException {
- // clear the buffers first. this part should not ever fail
- for (RecordDeserializer<?> deserializer : recordDeserializers) {
+ // release the deserializers first. this part should not ever
fail
+ for (int channelIndex = 0; channelIndex <
recordDeserializers.length; channelIndex++) {
+ releaseDeserializer(channelIndex);
+ }
+
+ // cleanup the barrier handler resources
+ barrierHandler.cleanup();
+ }
+
+ private void releaseDeserializer(int channelIndex) {
+ // recycle buffers and clear the deserializer.
+ RecordDeserializer<?> deserializer =
recordDeserializers[channelIndex];
+ if (deserializer != null) {
Review comment:
I tried to add the `@Nullable` annotate as you said, but the grammar didn't
support it, which reported the compilation error "'`@Nullable`' not applicable
to type use". In addition, that question on stackoverflow uses
`org.eclipse.jdt.annotation.NonNull` instead of the native annotate of JDK.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
