Hi Ritika, Thanks for your response. I still don’t like it, but then I don’t have to :) It’s clearly workable and really nice to get into 4.2.
AS1) Part of the RPC contract is that the new behaviour is enabled for TV >=2. In the section on Compatibility for New Brokers, please update the condition to “TV2 and later (value >= 2)”. AS2) OK, thanks for the extra information. AS3) There is still no point in using a tagged field here as far as I can see. Just use WriteTxnMarkersRequest v2 in AK 4.2 and later. I think we’d usually bump the version in a straightforward way for a change like this. Thanks, Andrew > On 20 Oct 2025, at 19:12, Ritika Reddy <[email protected]> wrote: > > Hey Andrew, > I think your concerns are very valid, however, I think this specific case > doesn't require too much tight coupling between this API and the TV. > 1) *What should it do if it receives TV3? What would the sender of TV3 > expect* > *What would the sender of TV3 expect the broker to do?* > A1) The write transaction markers request currently uses only the > transaction version to tighten the epoch check when epoch bumping is > supported at the end of every transaction. If this feature is reverted in > TV3 for any reason (which is highly unlikely), the feature owners would > need to enable explicit checks for that specific version anyway. It's more > likely that the newer versions will always want a stricter validation and > we want to allow smooth forward compatibility. > > 2) > > > *If a broker supports the V1 request, it can support TV2behaviour. If a > broker receives the V1 request, it knows that TV2 behaviouris expected. If > the coordinator sends the V1 request, it knows the recipient will honourTV2 > behaviour.* > A2) In this KIP, we are adding the transaction version per marker. A single > WriteTxnMarkersRequest can include multiple markers for different > transactions, possibly using different transaction versions. The epoch > check occurs per marker; therefore, even if the API request were V1, it's > not guaranteed that all markers require a TV2 check. We cannot map the API > version to the transaction version in this case, as it might cause > unintended effects. > > > > On Thu, Oct 16, 2025 at 6:41 AM Andrew Schofield <[email protected]> > wrote: > >> Hi Ritika, >> Thanks for your response. >> >> AS1: While I agree with the purpose of this KIP, I really do not agree >> with the detail. >> It seems to me that sending TransactionVersion in the request, and >> especially as >> a tagged field, is not the best way. I see the comment about trying to >> avoid an IBP >> bump, but I'm not convinced by the trade-off here. >> >> What if I have a broker that supports the new V1 WriteTxnMarkersRequest, >> but only >> supports TV2? What should it do if it receives TV3? What would the sender >> of TV3 expect >> the broker to do? If we ever introduce higher TV versions, all of this is >> going to come back >> to bite us. >> >> It seems to me that a better way is just to introduce V1 >> WriteTxnMarkersRequest with >> no schema change over V0. If a broker supports the V1 request, it can >> support TV2 >> behaviour. If a broker receives the V1 request, it knows that TV2 behaviour >> is expected. If the coordinator sends the V1 request, it knows the >> recipient will honour >> TV2 behaviour. >> >> In summary, I think it's much better to imply the TV in the protocol >> rather than send it. >> as an integer. It’s just as explicit in practice, but without the >> downsides of future >> compatibility. >> >> >> Hope this makes sense. >> Andrew >> >> >>> On 15 Oct 2025, at 23:12, Ritika Reddy <[email protected]> >> wrote: >>> >>> Hey Andrew, >>> >>> Thanks for the comment! The goal of this KIP is to check whether the >>> transaction version supports epoch bumps at the end of each transaction. >>> This feature is only supported in TV2 right now, not in TV0 / TV1. >>> >>> Since future versions are unlikely to remove the epoch bump feature, I >>> figured that it made sense to apply a stricter validation to all future >>> versions >= TV2. That keeps the logic consistent and avoids reintroducing >>> race conditions later. If a future version changes the behavior, we can >>> handle that explicitly at that point. I can add a disclaimer in the code >> to >>> double-check this part while introducing future versions. >>> >>> Let me know if that approach makes sense. >>> >>> On Wed, Oct 15, 2025 at 12:48 AM Andrew Schofield < >> [email protected]> >>> wrote: >>> >>>> Hi Ritika, >>>> Thanks for the KIP. I have one question. >>>> >>>> AS1: Does the new WriteTxnMarkers V1 request introduce support for >>>> TV == 2 or TV >= 2? I only ask because the pseudo-code is quite loose >>>> in the checking logic. I would have thought that V1 specifically >> introduces >>>> TV == 2 and it would take another version bump for this RPC to support >>>> higher transaction versions. >>>> >>>> Thanks, >>>> Andrew >>>> >>>>> On 15 Oct 2025, at 02:28, Artem Livshits <[email protected] >> .INVALID> >>>> wrote: >>>>> >>>>> Hi Ritika, >>>>> >>>>> Thanks for the KIP! The proposal makes sense to me. >>>>> >>>>> I have one clarification question, in the rejected alternatives the >> first >>>>> alternative says "However, its state is cleared after the first control >>>>> record is written". Is this the control record or the first batch? >>>>> >>>>> -Artem >>>>> >>>>> On Tue, Oct 14, 2025 at 10:39 AM Ritika Reddy >>>> <[email protected]> >>>>> wrote: >>>>> >>>>>> Hi All, >>>>>> I would like to start a discussion thread for KIP-1228. It's a >>>> relatively >>>>>> simple change and would really help strengthen EOS guarantees in >> Kafka. >>>>>> KIP Link - >>>>>> >>>>>> >>>> >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1228%3A+Add+Transaction+Version+to+WriteTxnMarkersRequest >>>>>> JIRA - https://issues.apache.org/jira/browse/KAFKA-19446 >>>>>> >>>>>> Thanks, >>>>>> Ritika >>>>>> >>>> >>>> >> >>
