Hi all, I do understand that everybody is excited about the Kafka 3.8 release, but this question is still actual :) Any updates.
Thanks, On Tue, Jul 16, 2024 at 2:54 PM dethmix dethmix <deth...@gmail.com> wrote: > Hi all, > > I am working on upgrading the Kafka cluster from the version 2.0.1 to > 3.5.2. The main concern in that update is passing version 2.1 which has the > following note in the upgrade document: > ``` > Note that 2.1.x contains a change to the internal schema used to store > consumer offsets. Once the upgrade is complete, it will not be possible to > downgrade to previous versions. > ``` > In the rolling upgrade section the documentation says the following: > ``` > Restart the brokers one by one for the new protocol version to take > effect. Once the brokers begin using the latest protocol version, it will > no longer be possible to downgrade the cluster to an older version. > ``` > So, as soon as inter.broker.protocol.version is set to 2.1+ you should not > be able to rollback to the old version. > I did reproduce the issue with the rollback with the following steps: > 1. Start cluster with Kafka 2.0.1 binaries and > inter.broker.protocol.version=2.0. Produce some messages in the test topic > 2. Restart cluster with Kafka 3.5.2 binaries > and inter.broker.protocol.version=3.5. Consume messages. > 3. Restart cluster once again with Kafka 2.0.1 binaries and > inter.broker.protocol.version=2.0. During start you will get error like > this: > ``` > ERROR [GroupMetadataManager brokerId=0] Error loading offsets from > __consumer_offsets-43 (kafka.coordinator.group.GroupMetadataManager) > org.apache.kafka.common.KafkaException: Unknown group metadata version 3 > ``` > > The interesting thing is that this doesn't happen if I rollback > inter.broker.protocol.version to 2.0 using Kafka 3.5.2 binaries. Cluster is > able to start and read all the data from the __consumer_offsets partitions > with metadata version 3. > The question is that expected behavior? Below are the steps that I have > used: > 1. Start cluster with Kafka 2.0.1 binaries and > inter.broker.protocol.version=2.0. Produce some messages in the test topic > 2. Restart cluster with Kafka 3.5.2 binaries > and inter.broker.protocol.version=3.5. Consume messages. > 3. Restart cluster once again with Kafka 3.5.2 binaries and > inter.broker.protocol.version=2.0. Cluster starts properly and is able to > read partitions from the __consumer_offsets topic. > > I did double check Kafka startup logs and make sure that it uses > configured inter.broker.protocol.version versions. > if this is okay, then I guess that can be used as a rollback in case of > problems after switching to the inter.broker.protocol.version 3.X > > Thanks in advance, > Dmytro >
