xBis7 opened a new pull request, #3741: URL: https://github.com/apache/ozone/pull/3741
## What changes were proposed in this pull request? In case there is a failure during a container delete there will be artifact leftovers on disk. To avoid that, containers will be renamed to a new location under a tmp directory and will be deleted from that location. This solution only works if SchemaV3 is enabled. For previous Schema versions, delete operation is exactly the same. To facilitate this implementation, a helper class called `CleanUpManager` was created which initializes the tmp directory on each disk under the path `/<volume>/<clusterId>/tmp/container_delete_service` and handles all operations related to this directory. Before every container delete, if SchemaV3 is enabled then we rename the container, set the path to the new location and then delete it from there. On datanode startup and shutdown we are checking the tmp directory for any leftovers and then clean it up. The datanode startup hook is in `VersionEndpointTask.checkVolumeSet()` and the shutdown is in `HddsDatanodeService.stop()`. ## What is the link to the Apache JIRA https://issues.apache.org/jira/browse/HDDS-6449 ## How was this patch tested? This patch was tested with new unit tests. We added `TestContainerPersistence.testDeleteContainerWithSchemaV3Enabled()` for testing `CleanUpManager` and `TestKeyValueHandler.testDeleteContainerWithSchemaV3Enabled()` for testing `KeyValueHandler.deleteInternal()`. In addition, we tested cleaning the tmp directory on datanode startup in `TestEndPoint.testGetVersionTask()` and on datanode shutdown in `TestHddsDatanodeService`. There were some issues with the existing code for `TestContainerPersistence.testDeleteContainer()`. The code block that was expected to throw an non-empty container exception, was never executed while the method was exiting upon the first exception thrown. Furthermore, the command that was used for deleting the container wasn’t checking if it was empty or not. In order to fix that, the method was splitted into two new ones and `KeyValueHandler.deleteContainer()` was used instead of `container.delete()`. Due to the changes related to SchemaV3 some tests under `container-service` package were failing, so we had to make small changes to fix their setups. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
