[
https://issues.apache.org/jira/browse/FLINK-16404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17164492#comment-17164492
]
Piotr Nowojski commented on FLINK-16404:
----------------------------------------
Hey [~zjwang] [~kevin.cyj], I think you have forgotten to remove this metric in
at least one remaining place [in the web
UI|https://github.com/apache/flink/blob/master/flink-runtime-web/web-dashboard/src/app/pages/job/checkpoints/subtask/job-checkpoints-subtask.component.html#L105].
Potentially we should also drop it from
{{TaskCheckpointStatisticsWithSubtaskDetails.CheckpointAlignment#CheckpointAlignment}}
- but this might affect our REST API and I'm not sure if the API is
{{@Public}} and stable?
> Avoid caching buffers for blocked input channels before barrier alignment
> -------------------------------------------------------------------------
>
> Key: FLINK-16404
> URL: https://issues.apache.org/jira/browse/FLINK-16404
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / Network
> Reporter: Zhijiang
> Assignee: Yingjie Cao
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.11.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> One motivation of this issue is for reducing the in-flight data in the case
> of back pressure to speed up checkpoint. The current default exclusive
> buffers per channel is 2. If we reduce it to 0 and increase somewhat floating
> buffers for compensation, it might cause deadlock problem because all the
> floating buffers might be requested away by some blocked input channels and
> never recycled until barrier alignment.
> In order to solve above deadlock concern, we can make some logic changes on
> both sender and receiver sides.
> * Sender side: It should revoke previous received credit after sending
> checkpoint barrier, that means it would not send any following buffers until
> receiving new credits.
> * Receiver side: The respective channel releases the requested floating
> buffers if barrier is received from the network. After barrier alignment, it
> would request floating buffers for the channels with positive backlog, and
> notify the sender side of available credits. Then the sender can continue
> transporting the buffers.
> Based on above changes, we can also remove the `BufferStorage` component
> completely, because the receiver would never reading buffers for blocked
> channels. Another possible benefit is that the floating buffers might be more
> properly made use of before barrier alignment.
> The only side effect would bring somehow cold setup after barrier alignment.
> That means the sender side has to wait for credit feedback to transport data
> just after alignment, which would impact on delay and network throughput. But
> considering the checkpoint interval not too short in general, so the above
> side effect can be ignored in practice. We can further verify it via existing
> micro-benchmark.
> After this ticket done, we still can not set exclusive buffers to zero ATM,
> there exists another deadlock issue which would be solved separately in
> another ticket.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
