[
https://issues.apache.org/jira/browse/FLINK-8523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16558825#comment-16558825
]
Nico Kruber commented on FLINK-8523:
------------------------------------
Sorry for coming back so late, but we recently discovered that it is actually
very important not to hold on to the received (floating and exclusive) buffers
during alignment, too, because then you would not use all available resources
to do the alignment for the remaining channels. This may make alignments
possibly worse than without flow control.
This ticket only prevents assigning new floating buffers to blocked channels
but the effect is somewhat limited. What we would really need is to also free
any floating buffers that we already have. Since we also have exclusive buffers
in the {{BarrierBuffer}} and need to keep the order, [~StephanEwen] and me were
thinking about 1) blocking the channel to not advertise more credit, 2)
spilling all of the already received buffers to disk (as before) and therefore
freeing all used floating buffers.
This ticket can be seen as a step towards that goal but I'd really like to see
the whole thing alltogether (possible follow-up tasks or change the ticket).
> Stop assigning floating buffers for blocked input channels in exactly-once
> mode
> -------------------------------------------------------------------------------
>
> Key: FLINK-8523
> URL: https://issues.apache.org/jira/browse/FLINK-8523
> Project: Flink
> Issue Type: Sub-task
> Components: Network
> Affects Versions: 1.5.0, 1.6.0
> Reporter: zhijiang
> Assignee: zhijiang
> Priority: Major
> Labels: pull-request-available
>
> In exactly-once mode, the input channel is set blocked state when reading
> barrier from it. And the blocked state will be released after barrier
> alignment or cancelled.
>
> In credit-based network flow control, we should avoid assigning floating
> buffers for blocked input channels because the buffers after barrier will not
> be processed by operator until alignment.
> To do so, we can fully make use of floating buffers and speed up barrier
> alignment in some extent.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
