Hey Justine and Divij, Thank you for the recommendations. I've made the updates to the KIP and added a new section called "Future Work: Update Message Format to Include Both Client Timestamp and LogAppend Timestamp."
Please take a look when get some time and let me know if there's anything else you'd like me to address. Thanks, Mehari On 6/5/23, 10:16 AM, "Divij Vaidya" <divijvaidy...@gmail.com <mailto:divijvaidy...@gmail.com>> wrote: CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe. Hey Justine Thank you for bringing this up. We had a discussion earlier in this [1] thread and concluded that bumping up the message version is a very expensive operation. Hence, we want to bundle together a bunch of impactful changes that we will perform on the message version and change it in v4.0. We are currently capturing the ideas here [2]. The idea of always having a log append time is captured in point 4 in the above wiki of ideas. As you suggested, we will add a new section called "future work" and add the idea of two timestamps (& why not do it now) over there. Meanwhile, does the above explanation answer your question on why not to do it right now? [1] https://lists.apache.org/thread/rxnps10t4vrsor46cx6xdj6t03qqxosh <https://lists.apache.org/thread/rxnps10t4vrsor46cx6xdj6t03qqxosh> [2] https://cwiki.apache.org/confluence/display/KAFKA/ideas+for+kafka+message+format+v.3 <https://cwiki.apache.org/confluence/display/KAFKA/ideas+for+kafka+message+format+v.3> -- Divij Vaidya On Mon, Jun 5, 2023 at 6:42 PM Justine Olshan <jols...@confluent.io.inva <mailto:jols...@confluent.io.inva>lid> wrote: > Hey Mehari, > Thanks for adding that section. I think one other thing folks have > considered is including two timestamps in the message format -- one for the > client side timestamp and one for the server side. Of course, this would > require a bump to the message format, and that hasn't happened in a while. > Could we include some information on this approach and why we aren't > pursuing it? I think message format bumps are tricky, but it is worth > calling out that this is also an option. > > Thanks, > Justine > > On Fri, Jun 2, 2023 at 4:51 PM Beyene, Mehari <meh...@amazon.com.inva > <mailto:meh...@amazon.com.inva>lid> > wrote: > > > Hi Justine, > > > > I added a section under Proposed Changes -> Timestamp Validation Logic to > > capture how the INVALID_TIMESTAMP is handled in this KIP. > > Please let me know if there are any additional areas you would like me to > > address. > > > > Thanks, > > Mehari > > > > > > > > > > >
