tanvipenumudy opened a new pull request, #6780: URL: https://github.com/apache/ozone/pull/6780
## What changes were proposed in this pull request? Recursive delete on the /tmp special volume, as well as its FSO/LEGACY buckets, fails using the ozone shell delete -r -y command with "java.lang.RuntimeException: Failed to clean bucket." Whenever we encounter the /tmp special volume, we call the `volume#deleteBucket` API and use `bucket#deleteKeys` to delete individual keys by iterating over the list of keys, instead of using the current `fs#delete` on the path. ## What is the link to the Apache JIRA https://issues.apache.org/jira/browse/HDDS-10881 ## How was this patch tested? Tested on a local docker-compose cluster. Volume creation using ozone sh: ``` bash-4.2$ ozone sh volume create tmp bash-4.2$ ozone sh bucket create tmp/dir1 bash-4.2$ ozone sh bucket info tmp/dir1 { "metadata" : { }, "volumeName" : "tmp", "name" : "dir1", "storageType" : "DISK", "versioning" : false, "listCacheSize" : 1000, "usedBytes" : 0, "usedNamespace" : 0, "creationTime" : "2024-06-03T11:58:22.160Z", "modificationTime" : "2024-06-03T11:58:22.160Z", "sourcePathExist" : true, "quotaInBytes" : -1, "quotaInNamespace" : -1, "bucketLayout" : "FILE_SYSTEM_OPTIMIZED", "owner" : "om", "link" : false } ``` Before Changes: ``` bash-4.2$ ozone sh bucket delete -r -y tmp/dir1 Could not delete bucket dir1. ``` After Changes: ``` bash-4.2$ ozone sh bucket delete -r -y tmp/dir1 bash-4.2$ bash-4.2$ ozone sh bucket info tmp/dir1 { "metadata" : { }, "volumeName" : "tmp", "name" : "dir1", "storageType" : "DISK", "versioning" : false, "listCacheSize" : 1000, "usedBytes" : 0, "usedNamespace" : 0, "creationTime" : "2024-06-03T11:58:22.160Z", "modificationTime" : "2024-06-03T11:58:22.160Z", "sourcePathExist" : true, "quotaInBytes" : -1, "quotaInNamespace" : -1, "bucketLayout" : "FILE_SYSTEM_OPTIMIZED", "owner" : "om", "link" : false } ``` Before Changes: ``` bash-4.2$ ozone sh volume delete -r -y tmp Exception in thread "pool-2-thread-1" java.lang.RuntimeException: Failed to clean bucket at org.apache.hadoop.ozone.shell.volume.DeleteVolumeHandler$BucketCleaner.run(DeleteVolumeHandler.java:202) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:829) VOLUME_NOT_EMPTY ``` After Changes: ``` bash-4.2$ ozone sh volume delete -r -y tmp bash-4.2$ bash-4.2$ ozone sh volume info tmp { "metadata" : { }, "name" : "tmp", "admin" : "om", "owner" : "om", "quotaInBytes" : -1, "quotaInNamespace" : -1, "usedNamespace" : 1, "creationTime" : "2024-06-03T11:58:02.705Z", "modificationTime" : "2024-06-03T11:58:02.705Z", "acls" : [ { "type" : "USER", "name" : "om", "aclScope" : "ACCESS", "aclList" : [ "ALL" ] }, { "type" : "GROUP", "name" : "om", "aclScope" : "ACCESS", "aclList" : [ "ALL" ] } ], "refCount" : 0 } ``` Volume creation using ozone fs: ``` bash-4.2$ ozone fs -mkdir ofs://omservice/tmp/ bash-4.2$ ozone sh volume info tmp { "metadata" : { }, "name" : "tmp", "admin" : "om", "owner" : "om", "quotaInBytes" : -1, "quotaInNamespace" : -1, "usedNamespace" : 1, "creationTime" : "2024-06-03T12:31:24.334Z", "modificationTime" : "2024-06-03T12:31:24.334Z", "acls" : [ { "type" : "USER", "name" : "om", "aclScope" : "ACCESS", "aclList" : [ "ALL" ] }, { "type" : "GROUP", "name" : "om", "aclScope" : "ACCESS", "aclList" : [ "ALL" ] } ], "refCount" : 0 } bash-4.2$ ozone sh bucket list tmp [ { "metadata" : { }, "volumeName" : "tmp", "name" : "d58da82289939d8c4ec4f40689c2847e", "storageType" : "DISK", "versioning" : false, "listCacheSize" : 1000, "usedBytes" : 0, "usedNamespace" : 0, "creationTime" : "2024-06-03T12:31:24.416Z", "modificationTime" : "2024-06-03T12:31:24.416Z", "sourcePathExist" : true, "quotaInBytes" : -1, "quotaInNamespace" : -1, "bucketLayout" : "FILE_SYSTEM_OPTIMIZED", "owner" : "om", "link" : false } ] bash-4.2$ ozone fs -touch ofs://omservice/tmp/sample.txt bash-4.2$ ozone fs -ls ofs://omservice/tmp/ Found 1 items -rw-rw-rw- 3 om om 0 2024-06-03 12:33 ofs://omservice/tmp/sample.txt ``` Before Changes: ``` bash-4.2$ ozone sh volume delete -r -y o3://omservice/tmp/ Exception in thread "pool-2-thread-1" java.lang.RuntimeException: Failed to clean bucket at org.apache.hadoop.ozone.shell.volume.DeleteVolumeHandler$BucketCleaner.run(DeleteVolumeHandler.java:202) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:829) VOLUME_NOT_EMPTY ``` After Changes: ``` bash-4.2$ ozone sh volume delete -r -y o3://omservice/tmp/ bash-4.2$ bash-4.2$ ozone fs -ls ofs://omservice/ Found 2 items drwxrwxrwx - om om 0 2024-06-03 12:30 ofs://omservice/s3v drwxrwxrwx - om om 0 2024-06-03 12:31 ofs://omservice/tmp bash-4.2$ ozone fs -ls ofs://omservice/tmp Found 1 items -rw-rw-rw- 3 om om 0 2024-06-03 12:33 ofs://omservice/tmp/sample.txt ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
