Hey Murali, Thanks for raising this.
I think it would make sense to take another look at the dependencies between the different CRUD objects that Kafka exposes, and add some sanity checks where they make sense and can be implemented safely. I was just speaking to a user that had some confusion for consumer groups referencing topics that had been deleted/truncated, and observing negative lag and non-delivery of the "rewritten" offsets. I wonder if we could improve the referential integrity there too. Thanks, Greg On Thu, Sep 19, 2024, 10:27 AM Muralidhar Basani <[email protected]> wrote: > Hello everyone, > > I’d like to propose an improvement to the Kafka topic deletion process, > specifically regarding ACLs. > > Currently, when we delete a topic using the kafka-topics.sh cli, the topic > is removed without any consideration for the ACLs that were applied to it. > This could lead to stale ACLs remaining in the system, which might clutter > the environment. > > Proposal: > Prevent Topic Deletion if ACLs Exist: If there are read or write ACLs > associated with the topic, the deletion should be prohibited by default. > This helps prevent accidental deletion of topics that are still in use. > > Force Delete Option: By introducing a --force option (e.g., -f), we allow > users to bypass the ACL check and delete the topic, even if ACLs are > present. > > Optional ACL Cleanup: Users could be provided with the option to delete > associated ACLs during the topic deletion process. However, this might not > apply to ACLs based on patterns, which would need to remain intact. > > Scenarios: > Scenario 1: No ACLs exist > > bin/kafka-topics.sh --bootstrap-server localhost:9092 --delete --topic > DummyTopic > Topic deleted. > > Scenario 2: ACLs exist > > bin/kafka-topics.sh --bootstrap-server localhost:9092 --delete --topic > DummyTopic > The following ACLs exist on the topic: > ResourcePattern(resourceType=TOPIC, name=DummyTopic, patternType=LITERAL, > ... operation=READ) > > Topic cannot be deleted. > > Scenario 3: ACLs exist, force delete > > bin/kafka-topics.sh --bootstrap-server localhost:9092 --delete --topic > DummyTopic -f > The following ACLs exist on the topic: > ResourcePattern(resourceType=TOPIC, name=DummyTopic, patternType=LITERAL, > ... operation=READ/WRITE) > > Topic is force deleted. > > Even though these acl resources are provisioned independent of a topic > existence, I see nice benefits : > > Accidental Deletion Prevention: This warning mechanism can help prevent > users from unintentionally deleting topics that are still in use. > Metadata Cleanup: It ensures a cleaner environment by avoiding stale ACLs > that may linger after topic deletion. > > I believe this is technically feasible, but I would appreciate hearing your > thoughts on it. and happy to open a KIP for it. > > Thanks, > Murali >
