When you get the exception and want to read from the first valid record
On Mon, Feb 12, 2018 at 12:09 PM, York Zhang <yorkz...@gmail.com> wrote:
> I'm trying to get the record count in a partition/topic.My solution is
> consuming all record in partition/topic and counting them with java api.
> I meet a problem that my topic is set retention.ms to 1 hour. When I
> consume the partition from offset whose record is already deleted, it seems
> that the offset is never added.I check group offset and find that the
> current-offset is always same as log-end-offset.
> Then I try to commit offset manually. If I get 0 records in a poll(), I
> will commit offset = old + 1 until I get some records. But it is too slow
> to reach the target.
> Is there any way that I can get the first valid record's offset in a
> partition? Then I can consume from that offset.
> I'm using kafka 0.9.