[
https://issues.apache.org/jira/browse/FLINK-14396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Piotr Nowojski closed FLINK-14396.
----------------------------------
> Implement rudimentary non-blocking network output
> -------------------------------------------------
>
> Key: FLINK-14396
> URL: https://issues.apache.org/jira/browse/FLINK-14396
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / Network
> Reporter: Zhijiang
> Assignee: Zhijiang
> Priority: Minor
> Labels: pull-request-available
> Fix For: 1.10.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Considering the mailbox model and unaligned checkpoints requirements in
> future, task network output should be non-blocking. In other words, as long
> as output is available, it should never block for a subsequent/future single
> record write.
> In the first version, we only implement the non-blocking output for the most
> regular case, and do not solve the following cases which still keep the
> previous behavior.
> * Big record which might span multiple buffers
> * Flatmap-like operators which might emit multiple records in every process
> * Broadcast watermark which might request multiple buffers at a time
> The solution is providing the RecordWriter#isAvailable method and respective
> LocalBufferPool#isAvailable for judging the output beforehand. As long as
> there is at-least one available buffer in LocalBufferPool, the RecordWriter
> is available for network output in most cases. This doesn’t include runtime
> handling of this non-blocking and availability behavior in
> StreamInputProcessor
> Note: It requires the minimum number of buffers in output LocalBufferPool
> adjusting to (numberOfSubpartitions + 1) and also adjusting the monitor of
> backpressure future.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
