Re: production ready for zookeeper to kraft migration
Matthieu, There are a few things to look out for during the migration. 1) Migration not starting. This is pretty much always going to be due to mis-configuration. Controller KRaftMigrationDriver logs should reveal what its waiting for 2) Migration of metadata fails. If some poison record in ZK causes the metadata migration to fail, you'll see stacktraces and ERRORs in the controller logs. The brokers will also not see an active controller and won't be getting metadata updates. 3) Something in the dual-write path fails. If the write back to ZK fails during dual-write mode (with ZK brokers or KRaft brokers), you'll see an increasing lag (ZkWriteBehindLag metric) So, in general: lack of an active controller, metadata not being propagated (e.g., ISRs not getting updated), errors in the logs. I'll try to get this written up in the docs soon. Cheers, David On Thu, Apr 4, 2024 at 12:56 PM Matthieu Patou wrote: > Hey Luke, > > Thank you for the update. > Out of curiosity if the migration is not working what are the symptoms ? is > it just that the controller won't show that the migration is complete ? or > could the controller claim (wrongfully) that the migration is complete when > it's not ? > > Best. > Matthieu > > On Wed, Apr 3, 2024 at 4:53 PM Luke Chen wrote: > > > Hi Matthieu, > > > > Yes, the ZK migrating to KRaft feature is already GA in v3.6.0. > > Sorry, we forgot to update the document in the Kafka-site repo. > > I've filed a PR for it: https://github.com/apache/kafka-site/pull/594 > > > > Thanks. > > Luke > > > > On Thu, Apr 4, 2024 at 6:14 AM Matthieu Patou > wrote: > > > > > I looked at the notes for 3.7.x and the migration from ZK to Kraft is > > still > > > not marked as production ready. > > > > > > I'm wondering what are the issues that people could be facing during > the > > > migration. > > > > > > If 4.0 is still planned to be the full removal for ZK, is there a plan > > for > > > something after 3.7 to mark ZK migration as production ready ? > > > > > > Best. > > > > > > Matthieu. > > > > > > -- -David
Re: production ready for zookeeper to kraft migration
Hey Luke, Thank you for the update. Out of curiosity if the migration is not working what are the symptoms ? is it just that the controller won't show that the migration is complete ? or could the controller claim (wrongfully) that the migration is complete when it's not ? Best. Matthieu On Wed, Apr 3, 2024 at 4:53 PM Luke Chen wrote: > Hi Matthieu, > > Yes, the ZK migrating to KRaft feature is already GA in v3.6.0. > Sorry, we forgot to update the document in the Kafka-site repo. > I've filed a PR for it: https://github.com/apache/kafka-site/pull/594 > > Thanks. > Luke > > On Thu, Apr 4, 2024 at 6:14 AM Matthieu Patou wrote: > > > I looked at the notes for 3.7.x and the migration from ZK to Kraft is > still > > not marked as production ready. > > > > I'm wondering what are the issues that people could be facing during the > > migration. > > > > If 4.0 is still planned to be the full removal for ZK, is there a plan > for > > something after 3.7 to mark ZK migration as production ready ? > > > > Best. > > > > Matthieu. > > >
Re: production ready for zookeeper to kraft migration
Hi Matthieu, Yes, the ZK migrating to KRaft feature is already GA in v3.6.0. Sorry, we forgot to update the document in the Kafka-site repo. I've filed a PR for it: https://github.com/apache/kafka-site/pull/594 Thanks. Luke On Thu, Apr 4, 2024 at 6:14 AM Matthieu Patou wrote: > I looked at the notes for 3.7.x and the migration from ZK to Kraft is still > not marked as production ready. > > I'm wondering what are the issues that people could be facing during the > migration. > > If 4.0 is still planned to be the full removal for ZK, is there a plan for > something after 3.7 to mark ZK migration as production ready ? > > Best. > > Matthieu. >
production ready for zookeeper to kraft migration
I looked at the notes for 3.7.x and the migration from ZK to Kraft is still not marked as production ready. I'm wondering what are the issues that people could be facing during the migration. If 4.0 is still planned to be the full removal for ZK, is there a plan for something after 3.7 to mark ZK migration as production ready ? Best. Matthieu.