[
https://issues.apache.org/jira/browse/STORM-886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14939794#comment-14939794
]
ASF GitHub Bot commented on STORM-886:
--------------------------------------
Github user revans2 commented on the pull request:
https://github.com/apache/storm/pull/700#issuecomment-144723285
@rsltrifork, No this does not solve that issue. The timeout is still a
hard coded value. The backpressure just means that the spout will not be
outputting new values. Old tuples can still time out and be replayed. The
problem here is the halting problem. How do you know that the bolt is waiting
in a controlled manner? Even if you do know that how do you know that if it is
waiting in a controlled manner that it has not lost track of a tuple? You have
to be able to predict the future to truly solve this problem. A hard coded
timeout is a simple solution. There have been a few other proposals to adjust
the timeout dynamically, but that all have potentially serious limitations
compared to a static timeout.
> Automatic Back Pressure
> -----------------------
>
> Key: STORM-886
> URL: https://issues.apache.org/jira/browse/STORM-886
> Project: Apache Storm
> Issue Type: Improvement
> Reporter: Robert Joseph Evans
> Assignee: Zhuo Liu
> Fix For: 0.11.0
>
> Attachments: aSimpleExampleOfBackpressure.png, backpressure.png
>
>
> This new feature is aimed for automatic flow control through the topology DAG
> since different components may have unmatched tuple processing speed.
> Currently, the tuples may get dropped if the downstream components can not
> process as quickly, thereby causing a waste of network bandwidth and
> processing capability. In addition, it is difficult to tune the
> max.spout.pending parameter for best backpressure performance. Therefore, an
> automatic back pressure scheme is highly desirable.
> Heron proposed a form of back pressure that does not rely on acking or max
> spout pending. Instead spouts throttle not only when max.spout.pending is
> hit, but also if any bolt has gone over a high water mark in their input
> queue, and has not yet gone below a low water mark again. There is a lot of
> room for potential improvement here around control theory and having spouts
> only respond to downstream bolts backing up, but a simple bang-bang
> controller like this is a great start.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
