pnowojski commented on a change in pull request #11507: [FLINK-16587] Add basic
CheckpointBarrierHandler for unaligned checkpoint
URL: https://github.com/apache/flink/pull/11507#discussion_r405550326
##########
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/api/writer/RecordWriter.java
##########
@@ -153,13 +153,17 @@ protected boolean copyFromSerializerToTargetChannel(int
targetChannel) throws IO
return pruneTriggered;
}
- public void broadcastEvent(AbstractEvent event) throws IOException {
+ public void broadcastEvent(AbstractEvent event, boolean
isPriorityEvent) throws IOException {
try (BufferConsumer eventBufferConsumer =
EventSerializer.toBufferConsumer(event)) {
for (int targetChannel = 0; targetChannel <
numberOfChannels; targetChannel++) {
tryFinishCurrentBufferBuilder(targetChannel);
- // Retain the buffer so that it can be recycled
by each channel of targetPartition
-
targetPartition.addBufferConsumer(eventBufferConsumer.copy(), targetChannel);
+ if (isPriorityEvent) {
+ targetPartition.addPriorityEvent(event,
targetChannel, eventBufferConsumer);
Review comment:
> My main motivation was to avoid unnecessary (de)serialization. In certain
cases (local input channel), you can directly pass the event.
We can safely ignore those optimisations for local channels. Events are
rare, that's one thing. Local channels are even more rare. And I agree that
having both parameters is a bit confusing. The same applies to the difference
how `bufferConsumer` is being copied.
> In general, I think it's up to the consumer to copy a buffer or not. If it
can immediately process the buffer, why copy and recycle at all?
Maybe, I'm not sure. Less efficient but more unified approach vs more
specialised one. As it is on the master, the contract is clear, callee has a
full ownership of the `bufferConsumer`. Always. With your proposal, some
branches/private methods would be behaving differently.
Either way it should be consistent across all of the public methods, so for
now I would be in favour of always copying and if you want, we could follow up
on this on trying out if changing the contract makes sense/is worth it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
