[
https://issues.apache.org/jira/browse/FLINK-16641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17342977#comment-17342977
]
Yingjie Cao commented on FLINK-16641:
-------------------------------------
Hi [~pnowojski], I have rebased the PR on the latest master branch and tested
the change by setting the default buffer per channel to 0, all tests passed (I
also ran several unaligned and aligned checkpoint it-cases multi-times
locally). Could you please take a look? I guess [~zjwang] do not have the time
to review it.
The change can be summarized as follows: (about 300 lines of change excluding
testing code)
In the existing implementation, the upstream task will announce backlog to the
downstream task and the downstream task will allocate buffers (credit) to
receive data from upstream. This PR enhances the current implementation in
three main aspects:
# If there is no initial credit, the upstream producer task will announce the
available backlog to the downstream consumer task when available data is
notified.
# The downstream consumer task will release all allocated buffers (credit) on
receiving the aligned checkpoint barrier. Besides, it will never allocate any
credit before checkpoint completion.
# For empty buffers of upstream task, instead of released directly, they will
be sent to the downstream task to release the buffers (credit) allocated for
these empty buffers.
> Announce sender's backlog to solve the deadlock issue without exclusive
> buffers
> -------------------------------------------------------------------------------
>
> Key: FLINK-16641
> URL: https://issues.apache.org/jira/browse/FLINK-16641
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / Network
> Reporter: Zhijiang
> Assignee: Yingjie Cao
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.14.0
>
>
> This is the second ingredient besides FLINK-16404 to solve the deadlock
> problem without exclusive buffers.
> The scenario is as follows:
> * The data in subpartition with positive backlog can be sent without doubt
> because the exclusive credits would be feedback finally.
> * Without exclusive buffers, the receiver would not request floating buffers
> for 0 backlog. But when the new backlog is added into such subpartition, it
> has no way to notify the receiver side without positive credits ATM.
> * So it would result in waiting for each other between receiver and sender
> sides to cause deadlock. The sender waits for credit to notify backlog and
> the receiver waits for backlog to request floating credits.
> To solve the above problem, the sender needs a separate message to announce
> backlog sometimes besides existing `BufferResponse`. Then the receiver can
> get this info to request floating buffers to feedback.
> The side effect brought is to increase network transport delay and throughput
> regression. We can measure how much it effects in existing micro-benchmark.
> It might probably bear this effect to get a benefit of fast checkpoint
> without exclusive buffers. We can give the proper explanations in respective
> configuration options to let users make the final decision in practice.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
