[
https://issues.apache.org/jira/browse/STORM-3058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16463675#comment-16463675
]
Stig Rohde Døssing commented on STORM-3058:
-------------------------------------------
Hi.
Please use English when raising issues.
Here's a quick Google translate:
{quote}
Spout in the nextTuple may pull more than one kafka message, will remain in the
_waitingToEmit, and then launch one by one, even if the
TOPOLOGY_MAX_SPOUT_PENDING set to 1, when the first tuple is defeated, the bolt
will still receive follow-up Message until _waitingToEmit is completely
transmitted.
This and I think TOPOLOGY_MAX_SPOUT_PENDING is not the same meaning, I hope
that the message can be strictly processed one by one, my topology worker
number, spout and bolt parallelism are set to 1
{quote}
TOPOLOGY_MAX_SPOUT_PENDING is only supposed to control how many tuples are
being processed in the topology at one time. The spout is free to fetch a
number of messages, as long as it doesn't emit more than one at a time.
It sounds like you're looking for a way to prevent the spout from fetching more
than one record at a time. I'm not sure you can do this with the storm-kafka
spout, but if you upgrade to storm-kafka-client, you can set the
max.poll.records property to 1, which will prevent the consumer from fetching
more than one record at a time.
> set TOPOLOGY_MAX_SPOUT_PENDING to one,but can get more than one tuple in bolt
> -----------------------------------------------------------------------------
>
> Key: STORM-3058
> URL: https://issues.apache.org/jira/browse/STORM-3058
> Project: Apache Storm
> Issue Type: Bug
> Components: storm-kafka
> Affects Versions: 0.10.2
> Reporter: [email protected]
> Priority: Major
>
> spout在nextTuple里可能会一次拉取到多条kafka消息,会保持到_waitingToEmit里,然后逐条发射,即使把TOPOLOGY_MAX_SPOUT_PENDING设置成1,当第一条tuple被fail时,bolt里仍然会收到后续的消息,直到_waitingToEmit全发射完。
> 这和我认为的TOPOLOGY_MAX_SPOUT_PENDING的意思是不是不一样呢,我希望消息能严格逐条处理,我的拓扑的worker数,spout和bolt的并行度都设置成1
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)