Hi, We have a storm topology that uses Kafka to read a topic with 6 partitions. ( Kafka 0.8.2, Storm 0.9.3 )
Recently, we had to set the KafkaSpout to read from the beginning, so we temporary configured our KafkaConfig this way: kafkaConfig.forceFromStart=true kafkaConfig.startOffsetTime = OffsetRequest.EarliestTime() It worked well, but afterward, setting those parameters back to false and to LatestTime respectively had no effect. In fact the topology won't read from our topic anymore. When the topology starts, The spout successully logs the offset and consumer group's cursor position for each partition in the worker log. But nothing is read. The only way we can read back from our Topic is to give our SpoutConfig a new Kafka ConsumerGroup Id in the SpoutConfig: ex: new SpoutConfig(zk, topic, zkStormRoot, newConsumerGroupID) Now, the only way I can see to read from the beginning would be to write the position we want to read from in Zookeeper where Consumer Group offset are stored and to restart our topology. I haven't tried it yet. Bottom line, it looks like as you as you use this forceFromStart flag, the consumer group id become unusable. Would anyone know if this is a bug in the KafkaSpout or an issue inherited from bug in Kafka? Thanks Francois
