[
https://issues.apache.org/jira/browse/STORM-756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15039169#comment-15039169
]
ASF GitHub Bot commented on STORM-756:
--------------------------------------
GitHub user HeartSaVioR opened a pull request:
https://github.com/apache/storm/pull/920
STORM-756 Handle taskids response as soon as possible (with fixing unit
test)
This PR fixes unit test which is included via #897
I've run the test from Travis CI 3 times and ShellBoltMessageQueueTest
always succeed.
@d2r @revans2 Please check it again. Thanks!
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/HeartSaVioR/storm STORM-756-v2-retry-1
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/storm/pull/920.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #920
----
commit 87f8fa1c559b8764715fd6f82f8524565d8398aa
Author: Jungtaek Lim <[email protected]>
Date: 2015-12-03T23:03:46Z
Revert "Merge branch 'revert-storm-756'"
This reverts commit ce2d49b924b6690d7704eccbb91ebbbea0a601bb, reversing
changes made to 18f68f7a967f7173b9c7c7444bedd53b13ad65fb.
commit f9890817884fa3c3592ee09e085c0cc58dc52149
Author: Jungtaek Lim <[email protected]>
Date: 2015-12-03T23:16:31Z
fix test: wait duration could be exactly 1000 ms
----
> [multilang] Introduce overflow control mechanism
> ------------------------------------------------
>
> Key: STORM-756
> URL: https://issues.apache.org/jira/browse/STORM-756
> Project: Apache Storm
> Issue Type: Improvement
> Components: storm-multilang
> Affects Versions: 0.10.0, 0.9.4, 0.11.0
> Reporter: Jungtaek Lim
> Assignee: Jungtaek Lim
>
> It's from STORM-738,
> https://issues.apache.org/jira/browse/STORM-738?focusedCommentId=14394106&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14394106
> A. ShellBolt side control
> We can modify ShellBolt to have sent tuple ids list, and stop sending tuples
> when list exceeds configured max value. In order to achieve this, subprocess
> should notify "tuple id is complete" to ShellBolt.
> * It introduces new commands for multi-lang, "proceed" (or better name)
> * ShellBolt stores in-progress-of-processing tuples list.
> * Its overhead could be big, subprocess should always notify to ShellBolt
> when any tuples are processed.
> B. subprocess side control
> We can modify subprocess to check pending queue after reading tuple.
> If it exceeds configured max value, subprocess can request "delay" to
> ShellBolt for slowing down.
> When ShellBolt receives "delay", BoltWriterRunnable should stop polling
> pending queue and continue polling later.
> How long ShellBolt wait for resending? Its unit would be "delay time" or
> "tuple count". I don't know which is better yet.
> * It introduces new commands for multi-lang, "delay" (or better name)
> * I don't think it would be introduced soon, but subprocess can request delay
> based on own statistics. (ex. pending tuple count * average tuple processed
> time for time unit, average pending tuple count for count unit)
> ** We can leave when and how much to request "delay" to user. User can make
> his/her own algorithm to control flooding.
> In my opinion B seems to more natural cause current issue is by subprocess
> side so it would be better to let subprocess overcome it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
