Hi Divij, Comments inline.
On Thu, Dec 21, 2023 at 7:00 AM Divij Vaidya <divijvaidy...@gmail.com> wrote: > 1\ Durability/availability bugs in kraft - Even though kraft has been > around for a while, we keep finding bugs that impact availability and data > durability in it almost with every release [1] [2]. It's a complex feature > and such bugs are expected during the stabilization phase. But we can't > remove the alternative until we see stabilization in kraft i.e. no new > stability/durability bugs for at least 2 releases. > I strongly disagree with this description. The first reference is in relation to a bug that affects zk mode too and has done since forever. The second issue was known and deemed not a blocker (and has since been fixed). KRaft mode is truly prod ready and is significantly better than zk mode based on prod metrics for thousands of clusters. 2\ Parity with Zk - There are also pending bugs [3] which are in the > category of Zk parity. Removing Zk from Kafka without having full feature > parity with Zk will leave some Kafka users with no upgrade path. > The bug you referenced isn't a real concern since it can be fixed in a patch release. > 3\ Test coverage - We also don't have sufficient test coverage for kraft > since quite a few tests are Zk only at this stage. > I disagree with this again, we have more than enough coverage for kraft. Given these concerns, I believe we need to reach 100% Zk parity and allow > new feature stabilisation (such as scram, JBOD) for at least 1 version > (maybe more if we find bugs in that feature) before we remove Zk. I also > agree with the point of view that we can't delay 4.0 indefinitely and we > need a clear cut line. > > Hence, I propose the following: > 1\ Keep trunk with 3.x. Release 3.8 and potentially 3.9 if we find major > (durability/availability related) bugs in 3.8. This will help users > continue to use their tried and tested Kafka setup until we have a proven > alternative from feature parity & stability point of view. > 2\ Release 4.0 as an "experimental" release along with 3.8 "stable" > release. This will help get user feedback on the feasibility of removing Zk > completely right now. > 3\ Create a criteria for moving 4.1 as "stable" release instead of > "experimental". This list should include 100% Zk parity and 100% Kafka > tests operating with kraft. It will also include other community feedback > from this & other threads. > 4\ When the 4.x version is "stable", move the trunk to 4.x and stop all > development on the 3.x branch. > Strong -1 to this proposal. I will send a separate email with what I think we should do. Ismael
