RE: How to get latest offsets with FlinkKafkaConsumer

, William From: Tzu-Li (Gordon) Tai [mailto:tzuli...@gmail.com] Sent: Friday, August 5, 2016 8:57 PM To: user@flink.apache.org Subject: Re: How to get latest offsets with FlinkKafkaConsumer Hi, Please also note that the “auto.offset.reset” property is only respected when there is no offsets

Re: How to get latest offsets with FlinkKafkaConsumer

Hi, Please also note that the “auto.offset.reset” property is only respected when there is no offsets under the same consumer group in ZK. So, currently, in order to make sure you read from the latest / earliest offsets every time you restart your Flink application, you’d have to use an unique

Re: How to get latest offsets with FlinkKafkaConsumer

Sorry, I think you are actually asking for the largest offset in the Kafka source, which makes it setProperty("auto.offset.reset", "largest"). > Am 05.08.2016 um 14:44 schrieb Stefan Richter : > > Hi, > > I think passing properties with

Re: How to get latest offsets with FlinkKafkaConsumer

Hi, I think passing properties with setProperty("auto.offset.reset", "smallest“) to the Kafka consumer should do what you want. Best, Stefan > Am 05.08.2016 um 14:36 schrieb Mao, Wei : > > I am doing some performance tests with Flink (1.0.3 )+ Kafka (0.8.2.2). And I >