zhijiangW commented on a change in pull request #10492: [FLINK-15140][runtime]
Fix shuffle data compression doesn't work with BroadcastRecordWriter.
URL: https://github.com/apache/flink/pull/10492#discussion_r356424126
##########
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/api/serialization/EventSerializer.java
##########
@@ -289,7 +289,7 @@ public static BufferConsumer
toBufferConsumer(AbstractEvent event) throws IOExce
MemorySegment data =
MemorySegmentFactory.wrap(serializedEvent.array());
- return new BufferConsumer(data, FreeingBufferRecycler.INSTANCE,
false);
+ return new BufferConsumer(data, FreeingBufferRecycler.INSTANCE,
false, true);
Review comment:
It is not always `true` for different cases calling `toBufferConsumer`. E.g.
`ResultSubpartition#finish()` to generate `EndOfPartitionEvent`.
So we can have two options:
- Provide the respective `isSharable` argument in the method of
`toBufferConsumer`. Then this tag in `BufferConsumer` is indicating for both
buffer and event.
- The `isSharable` field in `BufferConsumer` is only indicating for buffer
without event, then we do not need to touch this method to use default
constructor. It also makes sense because the compression is not working on
event ATM.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
