Hi all,

First of all thank you for providing such a great tool.

I have a question/problem and I would like to figure out exactly what is 
happening and workaround/fix it.
A bit of context:
- I have a Kafka Spout (the new one from storm-kafka-client) consuming a Kafka 
0.10 topic with 100 partitions
- It may happen that a Kafka consumer gets an offset that is out of range. In 
that case, the log message “Fetch offset <offset> is out of range for partition 
<topic-partiton>, resetting offset”
- Once this happen, the partition never gets consumed again and it seems like 
the new offset (either the latest or earliest depending on the consumer’s reset 
strategy) never gets committed.
- Re-submitting the topology does Not make the partition be consumed again

I am wondering if it is normal for the new offset to never get committed and 
the partition to never be consumed again. If yes, what would be the best 
workaround in order for the partition to get consumed again? It looks like 
enabling the auto-commit on the Kafka consumer helps, but that lose the 
reliability that Storm provides :-\

Thank you,

Teddy


Reply via email to