Thanks Gordon and Leonard I'm sorry that there is no specific case from my side, but I consider the issue as follows
1. Users may set an offset later than the current time because Flink does not limit it 2. If we use EARLIEST for a newly discovered partition with different OFFSETs, which may be different from the previous strategy. I think it's best to keep the same strategy as before if it does not cause data losing 3. I think support different OFFSETs in the FLIP will not make the implementation more complexity Of course, if it is confirmed that this is an illegal Timestamp OFFSET and Flink validate it. Then we can use the same strategy to apply to the newly discovered partition, I think this will be nice too Best, Shammon FY On Thu, Mar 30, 2023 at 12:29 PM Leonard Xu <xbjt...@gmail.com> wrote: > Thanks Hongshun and Shammon for driving the FLIP! > > > > *2. Clarification on "future-time" TIMESTAMP OffsetsInitializer* > > *3. Clarification on coupling SPECIFIC-OFFSET startup with > SPECIFIC-OFFSET > > post-startup* > > Grodan raised a good point about the future TIMESTAMP and SPECIFIC-OFFSET, > the timestamps/offset of the newly added partition is undetermined when the > job starts (the partition has not been created yet), and it is the > timestamps/offset in the future. > > I used many message queue systems like Kafka, Pulsar, xxMQ. In my past > experience, TIMESTAMP and SPECIFIC-OFFSET startup modes are usually used > to specify existing timestamps/offset, which are used for business > scenarios such as backfilling data and re-refreshing data. At present, It's > hard to imagine a user scenario specifying a future timestamp to filter > data in the current topic of message queue system. Is it overthinking to > consider future future TIMESTAMP and SPECIFIC-OFFSET? > > > Best, > Leonard
