[
https://issues.apache.org/jira/browse/KAFKA-16003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17796646#comment-17796646
]
Paolo Patierno commented on KAFKA-16003:
----------------------------------------
[~davidarthur] I did the following additional try ...
I decided to have the broker with auto.create.topics.enable=false.
Then when in dual-write mode, I created a my-topic-no-config (so without
providing any further configuration) and a my-topic-config (providing further
configuration, i.e. --config delete.retention.ms=10000). The result is that the
topic with further configuration appears in the /config/topics znode. Both
topics appear in the /brokers/topics.
The thing is that, when you rollback to ZooKeeper and you start a kafka console
producer, for example, it works fine for the my-topic-config topic (which was
created in KRaft dual-write with further configuration and lives in both
/config/topics and /brokers/topics znodes) but it doesn't work with the
my-topic-no-config (which was created with no further configuration so doesn't
appear in the /config/topics). The producer raises the usual error "[2023-12-14
10:49:03,811] WARN [Producer clientId=console-producer] Error while fetching
metadata with correlation id 6 : \{my-topic-1=UNKNOWN_TOPIC_OR_PARTITION}
(org.apache.kafka.clients.NetworkClient)" as the topic doesn't exist at all
(and of course cannot be created automatically). So I think this scenario
"brakes" somehow the cluster.
> The znode /config/topics is not updated during KRaft migration in
> "dual-write" mode
> -----------------------------------------------------------------------------------
>
> Key: KAFKA-16003
> URL: https://issues.apache.org/jira/browse/KAFKA-16003
> Project: Kafka
> Issue Type: Bug
> Components: controller
> Affects Versions: 3.6.1
> Reporter: Paolo Patierno
> Assignee: Mickael Maison
> Priority: Major
>
> I tried the following scenario ...
> I have a ZooKeeper-based cluster and create a my-topic-1 topic (without
> specifying any specific configuration for it). The correct znodes are created
> under /config/topics and /brokers/topics.
> I start a migration to KRaft but not moving forward from "dual write" mode.
> While in this mode, I create a new my-topic-2 topic (still without any
> specific config). I see that a new znode is created under /brokers/topics but
> NOT under /config/topics. It seems that the KRaft controller is not updating
> this information in ZooKeeper during the dual-write. The controller log shows
> that the write to ZooKeeper was done, but not everything I would say:
> {code:java}
> 2023-12-13 10:23:26,229 TRACE [KRaftMigrationDriver id=3] Create Topic
> my-topic-2, ID Macbp8BvQUKpzmq2vG_8dA. Transitioned migration state from
> ZkMigrationLeadershipState{kraftControllerId=3, kraftControllerEpoch=7,
> kraftMetadataOffset=445, kraftMetadataEpoch=7,
> lastUpdatedTimeMs=1702462785587, migrationZkVersion=236, controllerZkEpoch=3,
> controllerZkVersion=3} to ZkMigrationLeadershipState{kraftControllerId=3,
> kraftControllerEpoch=7, kraftMetadataOffset=445, kraftMetadataEpoch=7,
> lastUpdatedTimeMs=1702462785587, migrationZkVersion=237, controllerZkEpoch=3,
> controllerZkVersion=3}
> (org.apache.kafka.metadata.migration.KRaftMigrationDriver)
> [controller-3-migration-driver-event-handler]
> 2023-12-13 10:23:26,229 DEBUG [KRaftMigrationDriver id=3] Made the following
> ZK writes when handling KRaft delta: {CreateTopic=1}
> (org.apache.kafka.metadata.migration.KRaftMigrationDriver)
> [controller-3-migration-driver-event-handler] {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
