[
https://issues.apache.org/jira/browse/HDDS-8630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17723364#comment-17723364
]
Arun Sarin commented on HDDS-8630:
----------------------------------
[~sumitagrawl]
In both the workarounds mentioned above , we are not giving ours customers an
option to preserve & restore the trashed files.
Few scenarios that comes to mind :
Case 1 : Deleting & Preserving Huge Hierarchy directories and files will not be
supported
Case 2 : We would also see issues with Checkpointing. As it may create lot of
checkpoint directories and operation may fail
I think the ideal way would be to have an improvement task to find out the way
to skip quota for .trash folder . How currently we are handling this similar
scenario in case of quota for snapshot .
> Trash operation doesn't work as expected with Namespace quota set
> -----------------------------------------------------------------
>
> Key: HDDS-8630
> URL: https://issues.apache.org/jira/browse/HDDS-8630
> Project: Apache Ozone
> Issue Type: Bug
> Components: Ozone Manager
> Reporter: Arun Sarin
> Assignee: Sumit Agrawal
> Priority: Major
> Labels: proton
>
> Trash operation doesn't work as expected with Namespace quota set
> Scenario 1 :
> {code:java}
> [root@quasar-vavoom-1 ~]# ozone fs -rm ofs://ozone1/volume555/bucket555/file1
> 23/05/16 17:05:27 INFO Configuration.deprecation: io.bytes.per.checksum is
> deprecated. Instead, use dfs.bytes-per-checksum
> 23/05/16 17:05:27 WARN om.TrashPolicyOzone: Can't create trash directory:
> ofs://ozone1/volume555/bucket555/.Trash/hdfs/Current
> QUOTA_EXCEEDED org.apache.hadoop.ozone.om.exceptions.OMException: The
> namespace quota of Bucket:bucket555 exceeded: quotaInNamespace: 2 but
> namespace consumed: 4.
> at
> org.apache.hadoop.ozone.om.protocolPB.OzoneManagerProtocolClientSideTranslatorPB.handleError(OzoneManagerProtocolClientSideTranslatorPB.java:709)
> at
> org.apache.hadoop.ozone.om.protocolPB.OzoneManagerProtocolClientSideTranslatorPB.createDirectory(OzoneManagerProtocolClientSideTranslatorPB.java:1834)
> at
> org.apache.hadoop.ozone.client.rpc.RpcClient.createDirectory(RpcClient.java:1931)
> at
> org.apache.hadoop.ozone.client.OzoneBucket.createDirectory(OzoneBucket.java:764)
> at
> org.apache.hadoop.fs.ozone.BasicRootedOzoneClientAdapterImpl.createDirectory(BasicRootedOzoneClientAdapterImpl.java:512)
> at
> org.apache.hadoop.fs.ozone.BasicRootedOzoneFileSystem.mkdir(BasicRootedOzoneFileSystem.java:903)
> at
> org.apache.hadoop.fs.ozone.BasicRootedOzoneFileSystem.mkdirs(BasicRootedOzoneFileSystem.java:914)
> at
> org.apache.hadoop.ozone.om.TrashPolicyOzone.moveToTrash(TrashPolicyOzone.java:175)
> at org.apache.hadoop.fs.Trash.moveToTrash(Trash.java:110)
> at org.apache.hadoop.fs.Trash.moveToAppropriateTrash(Trash.java:96)
> at org.apache.hadoop.fs.shell.Delete$Rm.moveToTrash(Delete.java:153)
> at org.apache.hadoop.fs.shell.Delete$Rm.processPath(Delete.java:118)
> at
> org.apache.hadoop.fs.shell.Command.processPathInternal(Command.java:370)
> at org.apache.hadoop.fs.shell.Command.processPaths(Command.java:333)
> at
> org.apache.hadoop.fs.shell.Command.processPathArgument(Command.java:306)
> at org.apache.hadoop.fs.shell.Command.processArgument(Command.java:288)
> at org.apache.hadoop.fs.shell.Command.processArguments(Command.java:272)
> at
> org.apache.hadoop.fs.shell.FsCommand.processRawArguments(FsCommand.java:120)
> at org.apache.hadoop.fs.shell.Command.run(Command.java:179)
> at org.apache.hadoop.fs.FsShell.run(FsShell.java:328)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:81)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:95)
> at org.apache.hadoop.fs.ozone.OzoneFsShell.execute(OzoneFsShell.java:90)
> at
> org.apache.hadoop.fs.ozone.OzoneFsShell.lambda$main$0(OzoneFsShell.java:84)
> at
> org.apache.hadoop.hdds.tracing.TracingUtil.executeInNewSpan(TracingUtil.java:153)
> at org.apache.hadoop.fs.ozone.OzoneFsShell.main(OzoneFsShell.java:83)
> rm: Failed to move to trash: ofs://ozone1/volume555/bucket555/file1: The
> namespace quota of Bucket:bucket555 exceeded: quotaInNamespace: 2 but
> namespace consumed: 4.. Consider using -skipTrash option
> [root@quasar-vavoom-1 ~]# {code}
> Scenario 2
> {code:java}
> [root@quasar-vavoom-1 ~]# ozone fs -rm ofs://ozone1/volume555/bucket555/l1/l2
> 23/05/16 17:12:27 INFO Configuration.deprecation: io.bytes.per.checksum is
> deprecated. Instead, use dfs.bytes-per-checksum
> 23/05/16 17:12:27 WARN om.TrashPolicyOzone: Can't create trash directory:
> ofs://ozone1/volume555/bucket555/.Trash/hdfs/Current/l1
> QUOTA_EXCEEDED org.apache.hadoop.ozone.om.exceptions.OMException: The
> namespace quota of Bucket:bucket555 exceeded: quotaInNamespace: 2 but
> namespace consumed: 6.
> at
> org.apache.hadoop.ozone.om.protocolPB.OzoneManagerProtocolClientSideTranslatorPB.handleError(OzoneManagerProtocolClientSideTranslatorPB.java:709)
> at
> org.apache.hadoop.ozone.om.protocolPB.OzoneManagerProtocolClientSideTranslatorPB.createDirectory(OzoneManagerProtocolClientSideTranslatorPB.java:1834)
> at
> org.apache.hadoop.ozone.client.rpc.RpcClient.createDirectory(RpcClient.java:1931)
> at
> org.apache.hadoop.ozone.client.OzoneBucket.createDirectory(OzoneBucket.java:764)
> at
> org.apache.hadoop.fs.ozone.BasicRootedOzoneClientAdapterImpl.createDirectory(BasicRootedOzoneClientAdapterImpl.java:512)
> at
> org.apache.hadoop.fs.ozone.BasicRootedOzoneFileSystem.mkdir(BasicRootedOzoneFileSystem.java:903)
> at
> org.apache.hadoop.fs.ozone.BasicRootedOzoneFileSystem.mkdirs(BasicRootedOzoneFileSystem.java:914)
> at
> org.apache.hadoop.ozone.om.TrashPolicyOzone.moveToTrash(TrashPolicyOzone.java:175)
> at org.apache.hadoop.fs.Trash.moveToTrash(Trash.java:110)
> at org.apache.hadoop.fs.Trash.moveToAppropriateTrash(Trash.java:96)
> at org.apache.hadoop.fs.shell.Delete$Rm.moveToTrash(Delete.java:153)
> at org.apache.hadoop.fs.shell.Delete$Rm.processPath(Delete.java:118)
> at
> org.apache.hadoop.fs.shell.Command.processPathInternal(Command.java:370)
> at org.apache.hadoop.fs.shell.Command.processPaths(Command.java:333)
> at
> org.apache.hadoop.fs.shell.Command.processPathArgument(Command.java:306)
> at org.apache.hadoop.fs.shell.Command.processArgument(Command.java:288)
> at org.apache.hadoop.fs.shell.Command.processArguments(Command.java:272)
> at
> org.apache.hadoop.fs.shell.FsCommand.processRawArguments(FsCommand.java:120)
> at org.apache.hadoop.fs.shell.Command.run(Command.java:179)
> at org.apache.hadoop.fs.FsShell.run(FsShell.java:328)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:81)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:95)
> at org.apache.hadoop.fs.ozone.OzoneFsShell.execute(OzoneFsShell.java:90)
> at
> org.apache.hadoop.fs.ozone.OzoneFsShell.lambda$main$0(OzoneFsShell.java:84)
> at
> org.apache.hadoop.hdds.tracing.TracingUtil.executeInNewSpan(TracingUtil.java:153)
> at org.apache.hadoop.fs.ozone.OzoneFsShell.main(OzoneFsShell.java:83)
> rm: Failed to move to trash: ofs://ozone1/volume555/bucket555/l1/l2: The
> namespace quota of Bucket:bucket555 exceeded: quotaInNamespace: 2 but
> namespace consumed: 6.. Consider using -skipTrash option {code}
> RCA:
> Currently it is considering
> /.Trash/<user>/Current/ as 3 namespace + for every directory level inside the
> bucket is added as one [1]
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
