[
https://issues.apache.org/jira/browse/HDDS-8630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17723340#comment-17723340
]
Sumit Agrawal commented on HDDS-8630:
-------------------------------------
[~asarin] As discussed,
Trash also makes use of namespace quota as it will create a base trash
directory and other directory in hierarchy for keeping trash key. Since these
directory will also consume namespace, it will fail if quota is not enough.
These case will not normally happen in actual usecase, but it can happen when
deleting a set of directory having huge hierarchy and files. And need to
re-create these directory hierarchy again.
Options to handle these scenario:
# disable trash feature
# using --skipTrash as CLI option to avoid moving these file to trash
While comparing options with HDFS, few more options was available, which is not
present in ozone.
* skip quota for .trash folder
* move trash out to other folder
But these option are not available in ozone because, quota are not supported at
directory level for ozone.
And trash are configured with pre-defined path as per design of ozone trash
feature.
So these comparative feature are not available in ozone and these are not
required to be supported (as alternative is already available).
> Trash operation doesn't work as expected with Namespace quota set
> -----------------------------------------------------------------
>
> Key: HDDS-8630
> URL: https://issues.apache.org/jira/browse/HDDS-8630
> Project: Apache Ozone
> Issue Type: Bug
> Components: Ozone Manager
> Reporter: Arun Sarin
> Assignee: Sumit Agrawal
> Priority: Major
>
> Trash operation doesn't work as expected with Namespace quota set
> Scenario 1 :
> {code:java}
> [root@quasar-vavoom-1 ~]# ozone fs -rm ofs://ozone1/volume555/bucket555/file1
> 23/05/16 17:05:27 INFO Configuration.deprecation: io.bytes.per.checksum is
> deprecated. Instead, use dfs.bytes-per-checksum
> 23/05/16 17:05:27 WARN om.TrashPolicyOzone: Can't create trash directory:
> ofs://ozone1/volume555/bucket555/.Trash/hdfs/Current
> QUOTA_EXCEEDED org.apache.hadoop.ozone.om.exceptions.OMException: The
> namespace quota of Bucket:bucket555 exceeded: quotaInNamespace: 2 but
> namespace consumed: 4.
> at
> org.apache.hadoop.ozone.om.protocolPB.OzoneManagerProtocolClientSideTranslatorPB.handleError(OzoneManagerProtocolClientSideTranslatorPB.java:709)
> at
> org.apache.hadoop.ozone.om.protocolPB.OzoneManagerProtocolClientSideTranslatorPB.createDirectory(OzoneManagerProtocolClientSideTranslatorPB.java:1834)
> at
> org.apache.hadoop.ozone.client.rpc.RpcClient.createDirectory(RpcClient.java:1931)
> at
> org.apache.hadoop.ozone.client.OzoneBucket.createDirectory(OzoneBucket.java:764)
> at
> org.apache.hadoop.fs.ozone.BasicRootedOzoneClientAdapterImpl.createDirectory(BasicRootedOzoneClientAdapterImpl.java:512)
> at
> org.apache.hadoop.fs.ozone.BasicRootedOzoneFileSystem.mkdir(BasicRootedOzoneFileSystem.java:903)
> at
> org.apache.hadoop.fs.ozone.BasicRootedOzoneFileSystem.mkdirs(BasicRootedOzoneFileSystem.java:914)
> at
> org.apache.hadoop.ozone.om.TrashPolicyOzone.moveToTrash(TrashPolicyOzone.java:175)
> at org.apache.hadoop.fs.Trash.moveToTrash(Trash.java:110)
> at org.apache.hadoop.fs.Trash.moveToAppropriateTrash(Trash.java:96)
> at org.apache.hadoop.fs.shell.Delete$Rm.moveToTrash(Delete.java:153)
> at org.apache.hadoop.fs.shell.Delete$Rm.processPath(Delete.java:118)
> at
> org.apache.hadoop.fs.shell.Command.processPathInternal(Command.java:370)
> at org.apache.hadoop.fs.shell.Command.processPaths(Command.java:333)
> at
> org.apache.hadoop.fs.shell.Command.processPathArgument(Command.java:306)
> at org.apache.hadoop.fs.shell.Command.processArgument(Command.java:288)
> at org.apache.hadoop.fs.shell.Command.processArguments(Command.java:272)
> at
> org.apache.hadoop.fs.shell.FsCommand.processRawArguments(FsCommand.java:120)
> at org.apache.hadoop.fs.shell.Command.run(Command.java:179)
> at org.apache.hadoop.fs.FsShell.run(FsShell.java:328)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:81)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:95)
> at org.apache.hadoop.fs.ozone.OzoneFsShell.execute(OzoneFsShell.java:90)
> at
> org.apache.hadoop.fs.ozone.OzoneFsShell.lambda$main$0(OzoneFsShell.java:84)
> at
> org.apache.hadoop.hdds.tracing.TracingUtil.executeInNewSpan(TracingUtil.java:153)
> at org.apache.hadoop.fs.ozone.OzoneFsShell.main(OzoneFsShell.java:83)
> rm: Failed to move to trash: ofs://ozone1/volume555/bucket555/file1: The
> namespace quota of Bucket:bucket555 exceeded: quotaInNamespace: 2 but
> namespace consumed: 4.. Consider using -skipTrash option
> [root@quasar-vavoom-1 ~]# {code}
> Scenario 2
> {code:java}
> [root@quasar-vavoom-1 ~]# ozone fs -rm ofs://ozone1/volume555/bucket555/l1/l2
> 23/05/16 17:12:27 INFO Configuration.deprecation: io.bytes.per.checksum is
> deprecated. Instead, use dfs.bytes-per-checksum
> 23/05/16 17:12:27 WARN om.TrashPolicyOzone: Can't create trash directory:
> ofs://ozone1/volume555/bucket555/.Trash/hdfs/Current/l1
> QUOTA_EXCEEDED org.apache.hadoop.ozone.om.exceptions.OMException: The
> namespace quota of Bucket:bucket555 exceeded: quotaInNamespace: 2 but
> namespace consumed: 6.
> at
> org.apache.hadoop.ozone.om.protocolPB.OzoneManagerProtocolClientSideTranslatorPB.handleError(OzoneManagerProtocolClientSideTranslatorPB.java:709)
> at
> org.apache.hadoop.ozone.om.protocolPB.OzoneManagerProtocolClientSideTranslatorPB.createDirectory(OzoneManagerProtocolClientSideTranslatorPB.java:1834)
> at
> org.apache.hadoop.ozone.client.rpc.RpcClient.createDirectory(RpcClient.java:1931)
> at
> org.apache.hadoop.ozone.client.OzoneBucket.createDirectory(OzoneBucket.java:764)
> at
> org.apache.hadoop.fs.ozone.BasicRootedOzoneClientAdapterImpl.createDirectory(BasicRootedOzoneClientAdapterImpl.java:512)
> at
> org.apache.hadoop.fs.ozone.BasicRootedOzoneFileSystem.mkdir(BasicRootedOzoneFileSystem.java:903)
> at
> org.apache.hadoop.fs.ozone.BasicRootedOzoneFileSystem.mkdirs(BasicRootedOzoneFileSystem.java:914)
> at
> org.apache.hadoop.ozone.om.TrashPolicyOzone.moveToTrash(TrashPolicyOzone.java:175)
> at org.apache.hadoop.fs.Trash.moveToTrash(Trash.java:110)
> at org.apache.hadoop.fs.Trash.moveToAppropriateTrash(Trash.java:96)
> at org.apache.hadoop.fs.shell.Delete$Rm.moveToTrash(Delete.java:153)
> at org.apache.hadoop.fs.shell.Delete$Rm.processPath(Delete.java:118)
> at
> org.apache.hadoop.fs.shell.Command.processPathInternal(Command.java:370)
> at org.apache.hadoop.fs.shell.Command.processPaths(Command.java:333)
> at
> org.apache.hadoop.fs.shell.Command.processPathArgument(Command.java:306)
> at org.apache.hadoop.fs.shell.Command.processArgument(Command.java:288)
> at org.apache.hadoop.fs.shell.Command.processArguments(Command.java:272)
> at
> org.apache.hadoop.fs.shell.FsCommand.processRawArguments(FsCommand.java:120)
> at org.apache.hadoop.fs.shell.Command.run(Command.java:179)
> at org.apache.hadoop.fs.FsShell.run(FsShell.java:328)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:81)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:95)
> at org.apache.hadoop.fs.ozone.OzoneFsShell.execute(OzoneFsShell.java:90)
> at
> org.apache.hadoop.fs.ozone.OzoneFsShell.lambda$main$0(OzoneFsShell.java:84)
> at
> org.apache.hadoop.hdds.tracing.TracingUtil.executeInNewSpan(TracingUtil.java:153)
> at org.apache.hadoop.fs.ozone.OzoneFsShell.main(OzoneFsShell.java:83)
> rm: Failed to move to trash: ofs://ozone1/volume555/bucket555/l1/l2: The
> namespace quota of Bucket:bucket555 exceeded: quotaInNamespace: 2 but
> namespace consumed: 6.. Consider using -skipTrash option {code}
> RCA:
> Currently it is considering
> /.Trash/<user>/Current/ as 3 namespace + for every directory level inside the
> bucket is added as one [1]
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
