sumitagrawl commented on PR #4008: URL: https://github.com/apache/ozone/pull/4008#issuecomment-1331657916
> Thanks for the patch @sumitagrawl. > > Maybe I'm missing something, I got a question (maybe it's out of scope for this change). Why doesn't DirectoryDeletingService explore all the deleted files (recursively) and put items to deletedKeyTable by itself instead of taking the Ratis path to call `DirectoriesPurgeRequest`? The current process looks pretty cumbersome to me. > > I assume when a directory is deleted, the existence of the deleted directory in deletedDirectoryTable is reflected in all OM replicas, and so DirectoryDeletingService run with the same input per replica. As understanding, below are reason, 1. For HA, deletion of operation needs to be synchronized to other OM in cluster, so it make use of Ratis for this replication, so ratis path is taken. The rocks db handling is done in ratis for this purpose. 2. The task are done in batches to avoid resource consumed entirely for delete service, as batch can be too big. A configuration for handling limit in every iteration is present, default 100,000. Also Ratis synchronization is done with this limit. 3. The task include other things also, notifying SCM for delete of blocks which can consume time. So all these things are done in DirectoryDeletingService to reduce load in Raits flow which will be blocking other activity if taking time. @duongkame Please share if above make sense. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
