Hi Luke, Thanks for starting this discussion. I think it's very important that we communicate our plans and progress to send a clear message to users.
Regarding KRaft missing features, I tend to agree with Christopher that it would be much better to get them merged and declared production ready before 4.0. Otherwise users relying on them will not be able to upgrade to 4.0 and will have to wait. In my opinion, this would send a pretty negative message to these users. In terms of other changes planned for 4.0, what about the Log4j2 KIPs? We voted KIP-653 and KIP-719 and decided to wait for 4.0 to make the move. Finally we probably want a clear statement on the support plan for the 3.X series once 4.0 is out. Thanks, Mickael On Wed, Oct 11, 2023 at 6:45 PM Christopher Shannon <christopher.l.shan...@gmail.com> wrote: > > I think JBOD definitely needs to be before 4.0. That has been a blocker > issue this entire time for me and my team and I'm sure others. While Kraft > has been technically "production ready" for a while, I haven't been able to > upgrade because of missing JBOD support. > > On Wed, Oct 11, 2023 at 12:15 PM Ismael Juma <m...@ismaeljuma.com> wrote: > > > Hi Luke, > > > > This is a good discussion. And there is a lot more to it than KRaft. > > > > With regards to KRaft, there are two separate items: > > 1. Bugs > > 2. Missing features when compared to ZK > > > > When it comes to bugs, I don't see why 4.0 is particularly relevant. KRaft > > has been considered production-ready for over a year. If the bug is truly > > critical, we should fix it for 3.6.1 or 3.7.0 (depending on the > > complexity). > > > > When it comes to missing features, it would be preferable to land them > > before 4.0 as well (ideally 3.7). I believe KIP-858 (JBOD) is the obvious > > one in this category, but there are a few more in your list worth > > discussing. > > > > Ismael > > > > On Wed, Oct 11, 2023 at 5:18 AM Luke Chen <show...@gmail.com> wrote: > > > > > Hi all, > > > > > > While Kafka 3.6.0 is released, I’d like to start the discussion for the > > > “road to Kafka 4.0”. Based on the plan in KIP-833 > > > < > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-833%3A+Mark+KRaft+as+Production+Ready#KIP833:MarkKRaftasProductionReady-Kafka3.7 > > > >, > > > the next release 3.7 will be the final release before moving to Kafka 4.0 > > > to remove the Zookeeper from Kafka. Before making this major change, I'd > > > like to get consensus on the "must-have features/fixes for Kafka 4.0", to > > > avoid some users being surprised when upgrading to Kafka 4.0. The intent > > is > > > to have a clear communication about what to expect in the following > > months. > > > In particular we should be signaling what features and configurations are > > > not supported, or at risk (if no one is able to add support or fix known > > > bugs). > > > > > > Here is the JIRA tickets list > > > <https://issues.apache.org/jira/issues/?jql=labels%20%3D%204.0-blocker> > > I > > > labeled for "4.0-blocker". The criteria I labeled as “4.0-blocker” are: > > > 1. The feature is supported in Zookeeper Mode, but not supported in KRaft > > > mode, yet (ex: KIP-858: JBOD in KRaft) > > > 2. Critical bugs in KRaft, (ex: KAFKA-15489 : split brain in KRaft > > > controller quorum) > > > > > > If you disagree with my current list, welcome to have discussion in the > > > specific JIRA ticket. Or, if you think there are some tickets I missed, > > > welcome to start a discussion in the JIRA ticket and ping me or other > > > people. After we get the consensus, we can label/unlabel it afterwards. > > > Again, the goal is to have an open communication with the community about > > > what will be coming in 4.0. > > > > > > Below is the high level category of the list content: > > > > > > 1. Recovery from disk failure > > > KIP-856 > > > < > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-856:+KRaft+Disk+Failure+Recovery > > > >: > > > KRaft Disk Failure Recovery > > > > > > 2. Prevote to support controllers more than 3 > > > KIP-650 > > > < > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-650%3A+Enhance+Kafkaesque+Raft+semantics > > > >: > > > Enhance Kafkaesque Raft semantics > > > > > > 3. JBOD support > > > KIP-858 > > > < > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-858%3A+Handle+JBOD+broker+disk+failure+in+KRaft > > > >: > > > Handle > > > JBOD broker disk failure in KRaft > > > > > > 4. Scale up/down Controllers > > > KIP-853 > > > < > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-853%3A+KRaft+Controller+Membership+Changes > > > >: > > > KRaft Controller Membership Changes > > > > > > 5. Modifying dynamic configurations on the KRaft controller > > > > > > 6. Critical bugs in KRaft > > > > > > Does this make sense? > > > Any feedback is welcomed. > > > > > > Thank you. > > > Luke > > > > >
