[
https://issues.apache.org/jira/browse/STORM-394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rick Kellogg updated STORM-394:
-------------------------------
Component/s: storm-kafka
> Messages has expired, OFFSET_OUT_OF_RANGE, new offset startOffsetTime, no new
> messages, again and again
> -------------------------------------------------------------------------------------------------------
>
> Key: STORM-394
> URL: https://issues.apache.org/jira/browse/STORM-394
> Project: Apache Storm
> Issue Type: Bug
> Components: storm-kafka
> Affects Versions: 0.9.1-incubating
> Reporter: Vladislav Pernin
>
> Issue created here
> (https://github.com/wurstmeister/storm-kafka-0.8-plus/issues/55) but closed
> since the module is maintened under the Storm umbrella now.
> I think there might be a case that is not covered :
> 0) messages in Kafka has expired
> 1) so offset stored in Zookeeper are no longer valid
> 2) error OFFSET_OUT_OF_RANGE is thrown
> 3) getOffset with startOffsetTime
> 4) retry the fetch with the returned startOffset
> 5) get an ByteBufferMessageSet but empty
> KafkaUtils.fetchMessages seeems to be called again and again with the old
> offset and we get to step 2 again.
> I guess the new startOffset is not commited to Zookeeper since we do not have
> new messages.
> This can happen in the case of a topology restart, so it goes through the
> TridentKafkaEmitter.reEmitPartitionBatch
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
