[
https://issues.apache.org/jira/browse/STORM-3637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17175448#comment-17175448
]
Simon Cooper edited comment on STORM-3637 at 8/11/20, 10:52 AM:
----------------------------------------------------------------
However, it is currently possible to create a topology with that structure with
the current APIs
So either it needs to be disallowed at topology creation time, or it needs to
be made possible to use without deadlocking - as a minimum, by adding an option
to disable automatic backpressure. Obviously, my preference is for the latter,
as we have a genuine use of a looping topology structure currently in
production.
was (Author: thecoop1984):
However, it is very possible to create a topology with that structure with the
current APIs
So either it needs to be disallowed at topology creation time, or it needs to
be made possible to use without deadlocking - as a minimum, by adding an option
to disable automatic backpressure. Obviously, my preference is for the latter,
as we have a genuine use of a looping topology structure currently in
production.
> Looping topology structure can cause backpressure to deadlock
> -------------------------------------------------------------
>
> Key: STORM-3637
> URL: https://issues.apache.org/jira/browse/STORM-3637
> Project: Apache Storm
> Issue Type: Bug
> Components: storm-core
> Affects Versions: 2.0.0, 2.1.0
> Reporter: Simon Cooper
> Priority: Major
>
> When you have a topology structure with loops in it (BoltA and BoltB send
> tuples to each other), it can cause backpressure to deadlock.
> The scenario is that BoltA suddenly takes a long time to process a tuple (in
> our situation, it's doing a database operation). This causes the task input
> queue to fill up, setting the backpressure flag.
> BoltB, which is sending a tuple to BoltA, then cannot send, and the tuple is
> held in the emit queue. This blocks any tuples behind it, and also stops
> BoltB from executing. This means the input queue to BoltB will build up,
> until that backpressure flag is also set - and then when BoltA next wants to
> send a tuple to BoltB, it will irrevocably deadlock.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
