Xushaohong opened a new pull request, #3844:
URL: https://github.com/apache/ozone/pull/3844
## What changes were proposed in this pull request?
Currently, the deleting speed of paths in the FSO bucket is too slow.
Every time the DirectoryDeletingService executes(every 1m by default), it
chooses only one path from DeletedDirectroyTable and tries adding its sub-files
and sub-dirs into the PurgeDirectories Request. The real deletion happens in
OMKeyDeleteResponseWithFSO, which moved all sub-dirs into
DeletedDirectroyTable. Then these sub-dirs will be chosen by the future
execution of DirectoryDeletingService.
In the real production environment, such deletion speed is not applicable.
There could be lots of dirs deleted without sub-dirs or sub-files. The dir
deletion speed would lag behind the dir creation speed.
In this PR, I propose to optimize the logic of the
DirectoryDeletingService, we could consume all the quota
(ozone.path.deleting.limit.per.task) to delete paths as much as possible. The
good news is the PB is already designed to do so, we could add more
PurgePathRequest in one PurgeDirectoriesReques.
Originally: 1 min. At most 1 path from deletedDirectoryTable.
Now: 1min. At most 10000 paths from deletedDirectoryTable.
```
message PurgeDirectoriesRequest {
repeated PurgePathRequest deletedPath = 1;
}
```
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-7328
## How was this patch tested?
UT and real env
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
