[
https://issues.apache.org/jira/browse/APEXCORE-641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15861613#comment-15861613
]
ASF GitHub Bot commented on APEXCORE-641:
-----------------------------------------
GitHub user vrozov opened a pull request:
https://github.com/apache/apex-core/pull/470
APEXCORE-641 Subscribers/DataListeners may not be scheduled to execute even
when they have data to process
@PramodSSImmaneni, @tweise, @sandeshh Please review.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/vrozov/apex-core APEXCORE-641
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/apex-core/pull/470.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #470
----
commit aae4a97559c30c4b0ab610270b02489345996b5b
Author: Vlad Rozov <[email protected]>
Date: 2017-02-08T18:15:20Z
APEXCORE-641 Subscribers/DataListeners may not be scheduled to execute even
when they have data to process
----
> Subscribers/DataListeners may not be scheduled to execute even when they have
> data to process
> ---------------------------------------------------------------------------------------------
>
> Key: APEXCORE-641
> URL: https://issues.apache.org/jira/browse/APEXCORE-641
> Project: Apache Apex Core
> Issue Type: Bug
> Components: Buffer Server
> Affects Versions: 3.2.0, 3.3.0, 3.2.1, 3.4.0, 3.5.0, 3.6.0
> Reporter: Vlad Rozov
> Assignee: Vlad Rozov
>
> Buffer server iterates over DataListeners aka LogicalNodes and each
> LogicalNode tries to send to it's downstream all data that Publisher added to
> the DataList. When an output port is connected to multiple partitions or
> downstream operators (2 or more DataListeners/LogicalNodes) there may be more
> data published to the DataList after first few DataListeners in the listeners
> set iterated over DataList and reached the last block published so far. The
> data published while the last DataListeners sends data to it's downstream
> will not be processed by other DataListeners until Publisher adds more data
> to the DataList. This may lead to blocked operators, as Buffer server may
> stop processing data completely in case Publisher fills more than one Data
> block while a single DataListener sends data to it's downstream and there are
> more Subscribers/DataListeners than number of in memory blocks allowed (8).
> In such case, Publisher will be suspended, and there will be no task
> scheduled to process data already published to the DataList.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
