[ 
https://issues.apache.org/jira/browse/STORM-2957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16368374#comment-16368374
 ] 

Stig Rohde Døssing commented on STORM-2957:
-------------------------------------------

Please upgrade to 1.0.6. This issue, along with numerous others, should have 
been fixed in https://issues.apache.org/jira/browse/STORM-2937. Please let us 
know if it is still an issue in that version.

> Offsets are not committed when no ackers set (ackers=0) and KafkaSpout stops 
> polling messages once it reaches the max uncommitted offsets 
> ------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: STORM-2957
>                 URL: https://issues.apache.org/jira/browse/STORM-2957
>             Project: Apache Storm
>          Issue Type: Bug
>          Components: storm-kafka-client
>    Affects Versions: 1.0.5
>            Reporter: Ishwara Varnasi
>            Priority: Blocker
>
> A KafkaSpout stops polling messages from topic when topology has no ackers 
> set ("topology.acker.executors"= 0). This happens after KafkaSpout polls (and 
> emits) messages equal to the max uncommitted offsets 
> (KafkaspoutConfig.maxUncommittedOffsets).
> This happens because the KafkaSpout.ack() is called even before the emit 
> method adds the message to "emitted" collection (Set). In such cases, the 
> message is not added "acked" collection (Map), and hence never committed. 
> This seem to be bug introduced in a version later than storm-kafka-client 
> 1.0.1 and current code (1.0.5) has a check in ack() that checks if the acked 
> message is available in "emitted" collection and if not, ignores it (never 
> adds it to "acked" collection)
> Steps to reproduce issue:
> 1. Need a topology with KafkaSpout from storm-kafka-client
> 2. Set number of ackers ("topology.acker.executors") to o 
> (org.apache.storm.Config.setNumAckers(0)
> 3. While creating KafkaSpout instance, set 
> SpoutConfig.setMaxUncommittedOffsets to small number (50 or 100)
> 4. Start topology, see to it that KafkaSpout polls more messages from Kafka 
> that setMaxUncommittedOffsets
> 5. KafkaSpout stops polling messages from topic



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Reply via email to