Hi, I’m using 0.9.0.1 new-consumer api. I noticed that after kafka cleans up all old log segments(reach delete.retention time), I got unknown offset.
bin/kafka-consumer-groups.sh --bootstrap-server server:9092 --new-consumer --group testGroup --describe GROUP, TOPIC, PARTITION, CURRENT OFFSET, LOG END OFFSET, LAG, OWNER testGroup, test, 0, unknown, 49, unknown, consumer-1_/10.32.241.2 testGroup, test, 1, unknown, 61, unknown, consumer-1_/10.32.241.2 In this situation, I cannot consume anything using new-consumer java driver if I disable auto-commit. I think this happens because new-consumer driver stores offset in broker as a topic(not in zookeeper), and after reaching delete.retention time, it got deleted and becomes unknown. And since I disabled auto-commit, it can never know where it is, then it cannot consume anything. Is this what happened here? What should I do in this situation? Thanks, Jun
