Hi Rowland, KIP-939 provides a foundation for using a two-phase commit protocol with Kafka (allows it to be a participant) that can be used to implement various concrete protocols, such as XA but not only XA. The benefit of supporting a foundational construct (and not just one concrete protocol such as XA) is that it enables other implementations of two-phase commit to work with Kafka (e.g. Flink sync operator) and integrate with databases that don't support XA.
That said, XA is an important protocol that's supported by many databases and it makes sense to have an XA implementation for Kafka. The main consideration, I think, is whether we should build XA implementation into Kafka directly (which would require a KIP) or just provide an external utility that would implement XA based on KIP-939 (in this case it wouldn't change Kafka and wouldn't require a KIP or anyone's approvals). I'd be happy to collaborate to drive an XA solution forward either way. -Artem On Tue, Jan 2, 2024 at 1:06 PM Justine Olshan <jols...@confluent.io.invalid> wrote: > I believe Artem also had some conversations offline about XA. > > If I recall correctly, he didn't plan to include it in KIP-939 but was > happy to leave room for potential KIPs in the future. > Please feel free to continue the conversation on the thread. :) > > Justine > > On Tue, Jan 2, 2024 at 12:05 PM Greg Harris <greg.har...@aiven.io.invalid> > wrote: > > > Hi Rowland, > > > > First of all, welcome to the community, and thanks for thinking about > > the future of Kafka! > > > > I'm not very familiar with X/Open XA, but from the documentation I > > read, it appears most related to KIP-939: Support Participation in 2PC > > [1] currently in-progress. You may be interested in contributing to > > the discussion [2] for that KIP to ensure that it is easy to use > > within an XA context. I see that someone else in that thread has > > mentioned XA, but no conclusions appear to have been reached. I'm sure > > that Artem would be interested to hear your use-case and vision for > > using Kafka with XA. > > > > [1]: > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-939%3A+Support+Participation+in+2PC > > [2]: https://lists.apache.org/thread/wbs9sqs3z1tdm7ptw5j4o9osmx9s41nf > > > > Thanks! > > Greg > > > > > > On Mon, Jan 1, 2024 at 2:20 PM Rowland Smith <rowl...@gmail.com> wrote: > > > > > > Hi All, > > > > > > I am new to the Kafka developer community. After learning more about > > > Kafka's transactional capabilities recently, I became interested in > > > exploring what would be required to provide support for the XA > interface > > > specified in the X/ Open Distributed Processing Model in the Kafka > > producer > > > client. I have put together a proof of concept to satisfy my curiosity, > > and > > > based on that work, I think that extending the Kafka producer with XA > > > support is doable with reasonable effort. > > > > > > As I understand the Kafka development team's process, the first step in > > the > > > process would be to produce a KIP describing the feature's goals and > > > design. My question in this email is whether XA support has ever been > > > considered previously by the PMC and if so, with what result. I don't > > want > > > to spend time working on a KIP if XA support is not something that the > > PMC > > > sees value in including and supporting in the Kafka codebase. > > > > > > Any feedback would be appreciated. I am excited to work on this feature > > if > > > there is interest in the community. > > > > > > Regards, > > > Rowland > > >
