Hi, To clarify here, what I meant by "stuck" for deletion is regarding the state of LogDir deletion. When the cluster receives a topic deletion request, the KRAFT will delete the topic's metadata and eventually delete all LogDirs on any broker/disk at some point (correct me if I am wrong). My question is how to confirm the deletion state of these LogDirs. For example, what will happen for the following case?
- Topic-A has one partition with four replicas on broker1, 2, 3, 4 - The owner of Topic-A fired a topic deletion request. - Broker1,2,3 have no issues; however, the disk on broker 4 isn't reachable. - Broker1,2,3 deleted any LogDir they had for Topic-A. - Broker 4 can't delete the LogDir (for example, the disk went to read-only mode). - Now Topic-A's data isn't entirely deleted until this last LogDir on broker-4 is deleted. - For data compliance, we need to confirm the deletion of the data. Will KRAFT keeps listing the Topic-A until the last LogDir on broker-4 gets deleted? If this is the case, then this topic should be marked "Pending for deletion" for transparency; If KRAFT does not list the topic anymore, how will we confirm that all topic's LogDirs have been deleted? Thanks, Omnia On Tue, Nov 16, 2021 at 9:19 PM Colin McCabe <cmcc...@apache.org> wrote: > Hi Omnia, > > Topic deletion doesn't get stuck if a broker is down, when using KRaft. > There is no "deleting" state, only deleted or not deleted. > > best, > Colin > > On Tue, Nov 2, 2021, at 09:24, Omnia Ibrahim wrote: > > Hi Colin, thanks for your response. > > Regards your point that the topic gets deleted immediately, I got that we > > do this if the cluster is healthy. > > However, if there's a hardware failure with the disk or the broker is > > unreachable and has a replica; In these cases, deleting the log files > from > > the failed disk or unreachable broker will be impossible to delete until > we > > fix the hardware issue, > > So during troubleshooting, how will we know which topic is stuck for > > deletion because we can't delete some replicas because of hardware > > failures? > > > > Thanks > > > > > > On Mon, Nov 1, 2021 at 8:57 PM Colin McCabe <cmcc...@apache.org> wrote: > > > >> Hi Omnia, > >> > >> It is not necessary to know which topics are marked for deletion in when > >> in KRaft mode, because topic deletion happens immediately. > >> > >> best, > >> Colin > >> > >> On Thu, Oct 28, 2021, at 06:57, Omnia Ibrahim wrote: > >> > Hi, > >> > > >> > Kafka topicCommand used to report which topic is marked for deletion > by > >> > checking the znode on zookeeper; this feature has been deprecated > without > >> > replacement as part of KAFKA-12596 Remove deprecated --zookeeper in > >> > topicCommands <https://issues.apache.org/jira/browse/KAFKA-12596>. > >> > > >> > Also as far as I can see, there's no equivalent for this with KIP-500 > as > >> > well. > >> > > >> > Is there any other way to know the state of deletion for Kafka with ZK > >> and > >> > Without ZK? > >> > > >> > Is possible to leverage `RemoveTopicRecord` on the metadata topic to > >> > provide the same feature? > >> > > >> > > >> > Thanks > >> > > >> > Omnia > >> >
