Re: Did the semantics of Kafka's earliest offset change with the new source API?

2022-07-18 Thread Alexis Sarda-Espinosa
Hi David, thanks for the info, indeed I had misunderstood our old configuration, we didn't use earliest before, we just used the default. Regards, Alexis. Am Fr., 15. Juli 2022 um 14:46 Uhr schrieb David Anderson < dander...@apache.org>: > What did change was the default starting position when

Re: Did the semantics of Kafka's earliest offset change with the new source API?

2022-07-15 Thread David Anderson
What did change was the default starting position when not starting from a checkpoint. With FlinkKafkaConsumer, it starts from the committed offsets by default. With KafkaSource, it starts from the earliest offset. David On Fri, Jul 15, 2022 at 5:57 AM Chesnay Schepler wrote: > I'm not sure

Re: Did the semantics of Kafka's earliest offset change with the new source API?

2022-07-15 Thread Chesnay Schepler
I'm not sure about the previous behavior, but at the very least according to the documentation the behavior is identical. 1.12: https://nightlies.apache.org/flink/flink-docs-release-1.12/dev/connectors/kafka.html#kafka-consumers-start-position-configuration /|setStartFromEarliest()|///

Did the semantics of Kafka's earliest offset change with the new source API?

2022-07-13 Thread Alexis Sarda-Espinosa
Hello, I have a job running with Flink 1.15.0 that consumes from Kafka with the new KafkaSource API, setting a group ID explicitly and specifying OffsetsInitializer.earliest() as a starting offset. Today I restarted the job ignoring both savepoint and checkpoint, and the consumer started reading