zhijiangW commented on a change in pull request #10492: [FLINK-15140][runtime]
Fix shuffle data compression doesn't work with BroadcastRecordWriter.
URL: https://github.com/apache/flink/pull/10492#discussion_r355356824
##########
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/buffer/BufferConsumer.java
##########
@@ -44,6 +44,9 @@
private int currentReaderPosition;
+ /** Whether this BufferConsumer is copied from another BufferConsumer
instance. */
+ private final boolean isCopied;
Review comment:
I guess we can also support compression for broadcast mode in streaming job
via an more elegant way if have more time to think through, then we can avoid
bringing this tag field in `BufferConsumer`. That is why I suggested supporting
the compression only for batch jobs for nearly release, and make it also ready
for streaming job in future.
But I am not strongly objecting the current way if have no concerns for it.
@pnowojski
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
