[
https://issues.apache.org/jira/browse/APEXCORE-570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15823228#comment-15823228
]
Thomas Weise commented on APEXCORE-570:
---------------------------------------
Let’s first address the process issue (it may warrant a separate discussion and
additions to the contributor guidelines also). If you think there was a
conclusion then this may indicate that there was offline discussion that isn’t
captured here or anywhere else. Just by looking at this ticket it is everything
but clear what lead to your PR. This is not how the community can work,
discussion has to be in the open.
Now back to the topic: I think this is a good approach. It will avoid the fast
producing operator running ahead at the speed of writing to disk indefinitely.
With this we will not need to limit the spooling at all?
What about the case where you have two subscribers (and those could be
different operators) where one can keep up with the rate at which data is
published and the other one may be slow, albeit maybe temporarily? This will
slow down the fast subscriber and introduce latency.
> Prevent upstream operators from getting too far ahead when downstream
> operators are slow
> ----------------------------------------------------------------------------------------
>
> Key: APEXCORE-570
> URL: https://issues.apache.org/jira/browse/APEXCORE-570
> Project: Apache Apex Core
> Issue Type: Improvement
> Reporter: Pramod Immaneni
> Assignee: Pramod Immaneni
>
> If the downstream operators are slower than upstream operators then the
> upstream operators will get ahead and the gap can continue to increase.
> Provide an option to slow down or temporarily pause the upstream operators
> when they get too far ahead.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
