When you get the exception and want to read from the first valid record offset. Use, consumer#seekToBeginning(partitions)
https://kafka.apache.org/0100/javadoc/org/apache/kafka/clients/consumer/KafkaConsumer.html#seekToBeginning(java.util.Collection) On Mon, Feb 12, 2018 at 12:09 PM, York Zhang <yorkz...@gmail.com> wrote: > I'm trying to get the record count in a partition/topic.My solution is > consuming all record in partition/topic and counting them with java api. > > I meet a problem that my topic is set retention.ms to 1 hour. When I > consume the partition from offset whose record is already deleted, it seems > that the offset is never added.I check group offset and find that the > current-offset is always same as log-end-offset. > > Then I try to commit offset manually. If I get 0 records in a poll(), I > will commit offset = old + 1 until I get some records. But it is too slow > to reach the target. > > Is there any way that I can get the first valid record's offset in a > partition? Then I can consume from that offset. > > I'm using kafka 0.9. >
