Hey Justine Thank you for bringing this up. We had a discussion earlier in this [1] thread and concluded that bumping up the message version is a very expensive operation. Hence, we want to bundle together a bunch of impactful changes that we will perform on the message version and change it in v4.0. We are currently capturing the ideas here [2]. The idea of always having a log append time is captured in point 4 in the above wiki of ideas.
As you suggested, we will add a new section called "future work" and add the idea of two timestamps (& why not do it now) over there. Meanwhile, does the above explanation answer your question on why not to do it right now? [1] https://lists.apache.org/thread/rxnps10t4vrsor46cx6xdj6t03qqxosh [2] https://cwiki.apache.org/confluence/display/KAFKA/ideas+for+kafka+message+format+v.3 -- Divij Vaidya On Mon, Jun 5, 2023 at 6:42 PM Justine Olshan <jols...@confluent.io.invalid> wrote: > Hey Mehari, > Thanks for adding that section. I think one other thing folks have > considered is including two timestamps in the message format -- one for the > client side timestamp and one for the server side. Of course, this would > require a bump to the message format, and that hasn't happened in a while. > Could we include some information on this approach and why we aren't > pursuing it? I think message format bumps are tricky, but it is worth > calling out that this is also an option. > > Thanks, > Justine > > On Fri, Jun 2, 2023 at 4:51 PM Beyene, Mehari <meh...@amazon.com.invalid> > wrote: > > > Hi Justine, > > > > I added a section under Proposed Changes -> Timestamp Validation Logic to > > capture how the INVALID_TIMESTAMP is handled in this KIP. > > Please let me know if there are any additional areas you would like me to > > address. > > > > Thanks, > > Mehari > > > > > > > > > > >
