[
https://issues.apache.org/jira/browse/KAFKA-15605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Arthur updated KAFKA-15605:
---------------------------------
Description:
When migrating topics from ZooKeeper, the KRaft controller reads all the topic
and partition metadata from ZK directly. This includes topics which have been
marked for deletion by the ZK controller. After being migrated to KRaft, the
pending topic deletions are never completed, so it is as if the delete topic
request never happened.
Since the client request to delete these topics has already been returned as
successful, it would be confusing to the client that the topic still existed.
An operator or application would need to issue another topic deletion to remove
these topics once the controller had moved to KRaft. If they tried to create a
new topic with the same name, they would receive a TOPIC_ALREADY_EXISTS error.
The migration logic should carry over pending topic deletions and resolve them
either as part of the migration or shortly after.
To determine if a migration was affected by this, an operator can check the
contents of {{/admin/delete_topics}} after the KRaft controller has migrated
the metadata. If any topics are listed under this ZNode, they were not deleted
and will still be present in KRaft. At this point the operator can make a
determination if the topics should be re-deleted (using "kafka-topics.sh
--delete") or left in place. In either case, the topics should be removed from
{{/admin/delete_topics}} to prevent unexpected topic deletion in the event of a
fallback to ZK.
was:
When migrating topics from ZooKeeper, the KRaft controller reads all the topic
and partition metadata from ZK directly. This includes topics which have been
marked for deletion by the ZK controller. After being migrated to KRaft, the
pending topic deletions are never completed, so it is as if the delete topic
request never happened.
Since the client request to delete these topics has already been returned as
successful, it would be confusing to the client that the topic still existed.
An operator or application would need to issue another topic deletion to remove
these topics once the controller had moved to KRaft. If they tried to create a
new topic with the same name, they would receive a TOPIC_ALREADY_EXISTS error.
The migration logic should carry over pending topic deletions and resolve them
either as part of the migration or shortly after.
> Topics marked for deletion in ZK are incorrectly migrated to KRaft
> ------------------------------------------------------------------
>
> Key: KAFKA-15605
> URL: https://issues.apache.org/jira/browse/KAFKA-15605
> Project: Kafka
> Issue Type: Bug
> Components: controller, kraft
> Affects Versions: 3.6.0
> Reporter: David Arthur
> Priority: Major
> Fix For: 3.6.1
>
>
> When migrating topics from ZooKeeper, the KRaft controller reads all the
> topic and partition metadata from ZK directly. This includes topics which
> have been marked for deletion by the ZK controller. After being migrated to
> KRaft, the pending topic deletions are never completed, so it is as if the
> delete topic request never happened.
> Since the client request to delete these topics has already been returned as
> successful, it would be confusing to the client that the topic still existed.
> An operator or application would need to issue another topic deletion to
> remove these topics once the controller had moved to KRaft. If they tried to
> create a new topic with the same name, they would receive a
> TOPIC_ALREADY_EXISTS error.
> The migration logic should carry over pending topic deletions and resolve
> them either as part of the migration or shortly after.
> To determine if a migration was affected by this, an operator can check the
> contents of {{/admin/delete_topics}} after the KRaft controller has migrated
> the metadata. If any topics are listed under this ZNode, they were not
> deleted and will still be present in KRaft. At this point the operator can
> make a determination if the topics should be re-deleted (using
> "kafka-topics.sh --delete") or left in place. In either case, the topics
> should be removed from {{/admin/delete_topics}} to prevent unexpected topic
> deletion in the event of a fallback to ZK.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
