Prateek Agarwal created KAFKA-10646:
---------------------------------------
Summary: Support dynamic config for "delete.topic.enable"
Key: KAFKA-10646
URL: https://issues.apache.org/jira/browse/KAFKA-10646
Project: Kafka
Issue Type: Improvement
Components: core
Affects Versions: 1.1.2, 2.5.2
Reporter: Prateek Agarwal
Topic deletion in Kafka removes data and the operation is not reversible (i.e.
there is no "undelete" operation). Hence, we keep the flag
"delete.topic.enable" as "False" in our configs, so that someone accidentally
doesn't delete a topic, which can lead to incidents.
But sometimes, there are legit client use-cases, where they deprecate a topic
and those topics need to be deleted from the cluster. Currently, the process to
do this operation is very cumbersome:
{code:java}
1. Change the `server.properties` config on each broker with:
`delete.topic.enable=true`
2. Roll the cluster
3. Run Kafka Admin commands to delete the topics
4. Revert the `server.properties` config change for `delete.topic.enable`
5. Roll the cluster again{code}
There are other hacky workarounds as well, like:
{code:java}
1. Set Topic retention of the topic-to-be-deleted to a small value to flush the
data
2. Invoke `zkcli.sh rmr /brokers/topic/topic-to-be-deleted`
3. When Topic metadata is no longer available, rm the topic dirs on the
brokers{code}
The above process is pretty risky and can lead to unavailability of one or more
topics if any mistake happens in the commands.
Proposed solution:
{code:java}
Make the `delete.topic.enable` config dynamic so that the flag can be modified
without broker restarts {code}
Example runs:
1) Enable Topic Deletion Cluster wide
{code:java}
bin/kafka-configs.sh --bootstrap-server localhost:9092 \
--alter --add-config 'delete.topic.enable=true' \
--entity-default --entity-type brokers{code}
2) Enable Topic Deletion on Single Broker with broker ID: 0
{code:java}
bin/kafka-configs.sh --bootstrap-server localhost:9092 \
--alter --add-config 'delete.topic.enable=true' \
--entity-name 0 --entity-type brokers {code}
This will be similar to other broker dynamic configs currently available like
{{log.cleaner.threads}} which can be defined {{cluster-wide}} OR on a
per-broker level.
The precedence will look like so:
* Dynamic per-broker config stored in ZooKeeper
* Dynamic cluster-wide default config stored in ZooKeeper
* Static broker config from {{server.properties}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
