Re: [DISCUSS] KIP-868 Metadata Transactions (new thread)

2022-09-28 Thread David Arthur
Thanks Luke, I failed to mention in my reply to Ziming, but I did add this sentence to the end of the "visibility" section: The metadata shell must also avoid showing partially committed metadata > transactions. -David On Tue, Sep 27, 2022 at 10:20 PM Luke Chen wrote: > Hi David, > > I

Re: [DISCUSS] KIP-868 Metadata Transactions (new thread)

2022-09-27 Thread Luke Chen
Hi David, I think there's still an unanswered question from Ziming: > I think we can also describe visibility in the MetadataShell since it is also a public interface. I think this suggestion makes sense. But I'm still +1 for current proposal. Thanks. Luke On Tue, Sep 27, 2022 at 11:11 PM

Re: [DISCUSS] KIP-868 Metadata Transactions (new thread)

2022-09-27 Thread David Arthur
Thanks for the reviews, everyone. I’ve started a VOTE thread -David On Fri, Sep 23, 2022 at 00:55 deng ziming wrote: > David, > Thanks for the feedback about #2 and #3, I'm OK with them. > I also mentioned the visibility in the MetadataShell in #1, do you have any > thoughts? > > -- > Best, >

Re: [DISCUSS] KIP-868 Metadata Transactions (new thread)

2022-09-22 Thread deng ziming
David, Thanks for the feedback about #2 and #3, I'm OK with them. I also mentioned the visibility in the MetadataShell in #1, do you have any thoughts? -- Best, Ziming On Wed, Sep 21, 2022 at 10:56 PM David Arthur wrote: > Ziming, thanks for the feedback! Let me know your thoughts on #2 and #3

Re: [DISCUSS] KIP-868 Metadata Transactions (new thread)

2022-09-21 Thread David Arthur
Ziming, thanks for the feedback! Let me know your thoughts on #2 and #3 1. Good idea. I consolidated all the details of record visibility into that section. 2. I'm not sure we can always know the number of records ahead of time for a transaction. One future use case is likely for the ZK data

Re: [DISCUSS] KIP-868 Metadata Transactions (new thread)

2022-09-19 Thread deng ziming
Hello David, Thanks for the KIP, certainly it makes sense, I left some minor questions. 1. In “Record Visibility” section you declare visibility in the controller, in “Broker Support” you mention visibility in the broker, we can put them together, and I think we can also describe visibility in

Re: [DISCUSS] KIP-868 Metadata Transactions (new thread)

2022-09-19 Thread David Arthur
Thanks, Luke :) Colin -- I updated the KIP with your feedback. Do you think we would expose the "last stable offset" outside of the controller? Or would it just be an internal concept? -David On Sun, Sep 18, 2022 at 9:05 AM Luke Chen wrote: > Hi David, > > Thanks for the KIP! > It's a

Re: [DISCUSS] KIP-868 Metadata Transactions (new thread)

2022-09-18 Thread Luke Chen
Hi David, Thanks for the KIP! It's a light-weight transactional proposal for single producer, cool! +1 for it! Luke On Sat, Sep 10, 2022 at 1:14 AM David Arthur wrote: > Starting a new thread to avoid issues with mail client threading. > > Original thread follows: > > Hey folks, I'd like to

[DISCUSS] KIP-868 Metadata Transactions (new thread)

2022-09-09 Thread David Arthur
Starting a new thread to avoid issues with mail client threading. Original thread follows: Hey folks, I'd like to start a discussion on the idea of adding transactions in the KRaft controller. This will allow us to overcome the current limitation of atomic batch sizes in Raft which lets us do