[
https://issues.apache.org/jira/browse/STORM-886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Schonfeld updated STORM-886:
------------------------------------
Attachment: Screenshot 2015-11-08 22.17.57.png
> Automatic Back Pressure
> -----------------------
>
> Key: STORM-886
> URL: https://issues.apache.org/jira/browse/STORM-886
> Project: Apache Storm
> Issue Type: Improvement
> Components: storm-core
> Reporter: Robert Joseph Evans
> Assignee: Zhuo Liu
> Fix For: 0.11.0
>
> Attachments: Screenshot 2015-11-08 22.17.57.png, Screenshot
> 2015-11-08 22.18.06.png, aSimpleExampleOfBackpressure.png, backpressure.png
>
>
> This new feature is aimed for automatic flow control through the topology DAG
> since different components may have unmatched tuple processing speed.
> Currently, the tuples may get dropped if the downstream components can not
> process as quickly, thereby causing a waste of network bandwidth and
> processing capability. In addition, it is difficult to tune the
> max.spout.pending parameter for best backpressure performance. Another big
> motivation is that using max.spout.pending for flow control forces users to
> enable acking, which does not make sense.
> Therefore, an automatic back pressure scheme is highly desirable.
> Heron proposed a form of back pressure that does not rely on acking or max
> spout pending. Instead spouts throttle not only when max.spout.pending is
> hit, but also if any bolt has gone over a high water mark in their input
> queue, and has not yet gone below a low water mark again. There is a lot of
> room for potential improvement here around control theory and having spouts
> only respond to downstream bolts backing up, but a simple bang-bang
> controller like this is a great start.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
