Vladislav Pernin created STORM-394:
--------------------------------------
Summary: Messages has expired, OFFSET_OUT_OF_RANGE, new offset
startOffsetTime, no new messages, again and again
Key: STORM-394
URL: https://issues.apache.org/jira/browse/STORM-394
Project: Apache Storm (Incubating)
Issue Type: Bug
Affects Versions: 0.9.1-incubating
Reporter: Vladislav Pernin
Issue created here
(https://github.com/wurstmeister/storm-kafka-0.8-plus/issues/55) but closed
since the module is maintened under the Storm umbrella now.
I think there might be a case that is not covered :
0) messages in Kafka has expired
1) so offset stored in Zookeeper are no longer valid
2) error OFFSET_OUT_OF_RANGE is thrown
3) getOffset with startOffsetTime
4) retry the fetch with the returned startOffset
5) get an ByteBufferMessageSet but empty
KafkaUtils.fetchMessages seeems to be called again and again with the old
offset and we get to step 2 again.
I guess the new startOffset is not commited to Zookeeper since we do not have
new messages.
This can happen in the case of a topology restart, so it goes through the
TridentKafkaEmitter.reEmitPartitionBatch
--
This message was sent by Atlassian JIRA
(v6.2#6252)
