[ 
https://issues.apache.org/jira/browse/STORM-3058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16463675#comment-16463675
 ] 

Stig Rohde Døssing commented on STORM-3058:
-------------------------------------------

Hi.

Please use English when raising issues.

Here's a quick Google translate:

{quote}
Spout in the nextTuple may pull more than one kafka message, will remain in the 
_waitingToEmit, and then launch one by one, even if the 
TOPOLOGY_MAX_SPOUT_PENDING set to 1, when the first tuple is defeated, the bolt 
will still receive follow-up Message until _waitingToEmit is completely 
transmitted.

This and I think TOPOLOGY_MAX_SPOUT_PENDING is not the same meaning, I hope 
that the message can be strictly processed one by one, my topology worker 
number, spout and bolt parallelism are set to 1
{quote}

TOPOLOGY_MAX_SPOUT_PENDING is only supposed to control how many tuples are 
being processed in the topology at one time. The spout is free to fetch a 
number of messages, as long as it doesn't emit more than one at a time.

It sounds like you're looking for a way to prevent the spout from fetching more 
than one record at a time. I'm not sure you can do this with the storm-kafka 
spout, but if you upgrade to storm-kafka-client, you can set the 
max.poll.records property to 1, which will prevent the consumer from fetching 
more than one record at a time.

> set TOPOLOGY_MAX_SPOUT_PENDING to one,but can get more than one tuple in bolt
> -----------------------------------------------------------------------------
>
>                 Key: STORM-3058
>                 URL: https://issues.apache.org/jira/browse/STORM-3058
>             Project: Apache Storm
>          Issue Type: Bug
>          Components: storm-kafka
>    Affects Versions: 0.10.2
>            Reporter: [email protected]
>            Priority: Major
>
> spout在nextTuple里可能会一次拉取到多条kafka消息,会保持到_waitingToEmit里,然后逐条发射,即使把TOPOLOGY_MAX_SPOUT_PENDING设置成1,当第一条tuple被fail时,bolt里仍然会收到后续的消息,直到_waitingToEmit全发射完。
> 这和我认为的TOPOLOGY_MAX_SPOUT_PENDING的意思是不是不一样呢,我希望消息能严格逐条处理,我的拓扑的worker数,spout和bolt的并行度都设置成1



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Reply via email to