How can I get the first valid record's offset in a partition?

2018-02-11 Thread York Zhang
I'm trying to get the record count in a partition/topic.My solution is consuming all record in partition/topic and counting them with java api. I meet a problem that my topic is set retention.ms to 1 hour. When I consume the partition from offset whose record is already deleted, it seems that

How to set offset of a (topic,partition,group) without closing other consumers?

2018-03-12 Thread York Zhang
I have some running consumers. I want to set the offset of a specific topic-partition-group with a new consumer instance in the same group. I have tried to use subscribe(), but kafka randomly assign partition to the new consumer, I can't operate the partition I want. Then I tried to use assign(),