You are right that there's currently a bug, there's a pending fix here https://github.com/apache/storm/pull/2181. Feedback on it is appreciated :)
2017-09-18 7:34 GMT+02:00 chandan singh <cks07...@gmail.com>: > Hi > > I wanted to get my understanding about CommitOffset/FetchOffset in > KafkaSout (v1.1.1) confirmed. > > For Kafka consumer, commitOffset and fetchOffset are same. In other words, > Kafka consumer reads from last committedOffset and hence their official > documentation ( > https://kafka.apache.org/0110/javadoc/index.html?org/apache/ > kafka/clients/consumer/KafkaConsumer.html) > has the following note in bold "*Note: The committed offset should always > be the offset of the next message that your application will read.*" > > KafkaSpout does not follow that guideline. Obviously, what KafkaSpout does, > by treating commitOffset as one less than the next fetchOffset, sounds much > better but this makes some kafka tools misbehave with offsets committed by > KafkaSpout. Also, a migration from custom spout to KafkaSpout becomes less > trivial. > > Please confirm if my understanding is correct and leave your comments. > > Regards > Chandan >
