Tan-JiaLiang commented on PR #52: URL: https://github.com/apache/flink-connector-kafka/pull/52#issuecomment-1751898387
@tzulitai Maybe this will lead to a new bug. Assume topic has 3 partitions, A/B/C tp=0: Write a record to each partitions. All of partitions get a new record. The offset of A=1,B=1,C=1. tp=1: Start job with latest-offsets offsets initializer. tp=2: Write a record to partition A and B, C is empty. tp=3: Stop the job with savepoint. We save the `currentOffset` A=2,B=2,C=-1 into savepoint. tp=4: Restore the job from savepoint. After restore, partition A and B's consume is ok, but because of currentOffset C=-1, it will consume with `seekToBeginning`, so data offset=0 will be consumed, which should not to be. But after your proposal, I think i'm not considering the compatibility of the state, `currentOffset` will error in restore when it equals to -1, which will cause confused for users after they upgrade the connector. I will refine the above test case and state compatibility problem if need after we discuss a result. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
