[
https://issues.apache.org/jira/browse/FLINK-2497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17328976#comment-17328976
]
Flink Jira Bot commented on FLINK-2497:
---------------------------------------
This major issue is unassigned and itself and all of its Sub-Tasks have not
been updated for 30 days. So, it has been labeled "stale-major". If this ticket
is indeed "major", please either assign yourself or give an update. Afterwards,
please remove the label. In 7 days the issue will be deprioritized.
> Rework streaming iteration flow control
> ---------------------------------------
>
> Key: FLINK-2497
> URL: https://issues.apache.org/jira/browse/FLINK-2497
> Project: Flink
> Issue Type: Improvement
> Components: API / DataStream
> Reporter: Gyula Fora
> Priority: Major
> Labels: stale-major
>
> Currently streaming iterations use a simple Blocking queue to pass data from
> the iteration tails to the iteration heads.
> While this works most of the time, during increased load (or even short
> bursts) the increased amount of records in transit inside the loop can cause
> the limited number of input/output buffer-pools to fill up and deadlock the
> system. (caused by the back-pressure mechanism)
> One solution to this problem could be to remove back-pressure on the
> loop-edge by replacing the Blocking queue with some soft spilling queue that
> will write the records automatically to disk letting the iteration tail to
> progress with its input.
> For a more robust solution we need to implement more elaborate flow control
> in streaming programs containing iterations.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
