Re: [DISCUSS] KIP-865 Metadata Transactions

2022-09-09 Thread David Arthur
Starting a new thread here https://lists.apache.org/thread/895pgb85l08g2l63k99cw5dt2qpjkxb9 On Fri, Sep 9, 2022 at 1:05 PM Colin McCabe wrote: > > Also, it looks like someone already claimed KIP-865, so I'd suggest grabbing > a new number. :) > > Colin > > > On Fri, Sep 9, 2022, at 09:38, Colin

Re: [DISCUSS] KIP-865 Metadata Transactions

2022-09-09 Thread Colin McCabe
Also, it looks like someone already claimed KIP-865, so I'd suggest grabbing a new number. :) Colin On Fri, Sep 9, 2022, at 09:38, Colin McCabe wrote: > Thanks for this KIP, David! > > In the "motivation" section, it might help to give a concrete example > of an operation we want to be atomic.

Re: [DISCUSS] KIP-865 Metadata Transactions

2022-09-09 Thread Николай Ижиков
Hello. Sorry, but it seems we should fix KIP numbering. KIP-865 [1] is another KIP :). [1] https://cwiki.apache.org/confluence/display/KAFKA/KIP-865%3A+Support+--bootstrap-server+in+kafka-streams-application-reset > 9 сент. 2022 г., в 19:38, Colin McCabe написал(а): > > Thanks for this KIP, D

Re: [DISCUSS] KIP-865 Metadata Transactions

2022-09-09 Thread Colin McCabe
Thanks for this KIP, David! In the "motivation" section, it might help to give a concrete example of an operation we want to be atomic. My favorite one is probably CreateTopics since it's easy to see that we want to create all of a topic or none of it, and a topic could be a potentially unbound

[DISCUSS] KIP-865 Metadata Transactions

2022-09-09 Thread David Arthur
Hey folks, I'd like to start a discussion on the idea of adding transactions in the KRaft controller. This will allow us to overcome the current limitation of atomic batch sizes in Raft which lets us do things like create topics with a huge number of partitions. https://cwiki.apache.org/confluence