[
https://issues.apache.org/jira/browse/STORM-1358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15252218#comment-15252218
]
Nisarg Shah commented on STORM-1358:
------------------------------------
Hey,
I am looking at things in open source to contribute to. I'll be looking to do
this one.
> Porting JStorm multi-thread mode of spout to Storm
> --------------------------------------------------
>
> Key: STORM-1358
> URL: https://issues.apache.org/jira/browse/STORM-1358
> Project: Apache Storm
> Issue Type: New Feature
> Reporter: Basti Liu
> Assignee: Basti Liu
> Labels: jstorm-merger
>
> There are two modes of spout, "single-thread" and "multi-thread" in JStorm.
> The "single-thread" mode is simliar to Storm while the "multi-thread" mode
> separates the processing of ack/fail and nextTuple to two threads. It means
> we can stay in nextTuple for a long time without any side effect on ack/fail.
> Let's think about an example of kafka spout. We can initiate a consumer
> thread for kafka when initialization of spout. Then the comsumer starts to
> pull events from kafka and pulish the retreived events into a local queue. At
> meantime, nextTuple waits to read at this queue. If any available events,
> nextTuple will get notification faster and flush them to downstream. This
> model could probably introduce better performance compared with
> "single-thread" mode.
> For this mode, the max pending configuration of spout might not be useful as
> expectation. It depends on how long we stay in nextTuple. But backpressure is
> a good choice to resolve flow control problem.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
