I'm trying to get the record count in a partition/topic.My solution is consuming all record in partition/topic and counting them with java api.
I meet a problem that my topic is set retention.ms to 1 hour. When I consume the partition from offset whose record is already deleted, it seems that the offset is never added.I check group offset and find that the current-offset is always same as log-end-offset. Then I try to commit offset manually. If I get 0 records in a poll(), I will commit offset = old + 1 until I get some records. But it is too slow to reach the target. Is there any way that I can get the first valid record's offset in a partition? Then I can consume from that offset. I'm using kafka 0.9.
