[
https://issues.apache.org/jira/browse/HADOOP-15575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16529752#comment-16529752
]
Steve Loughran commented on HADOOP-15575:
-----------------------------------------
You could do all of this with a custom trash policy; add an S3A aware one which
asked the FS bucket conf, delegating to the default trash policy for all other
stores.
You'd want to override {{Trash.getEmptier()}} to return a new Emptier...this
would be instantiated in the NN of any (transient) HDFS cluster, so still need
to do the normal cleanup for HDFS. As it never gets pointed at the object store
(presumably), it's not doing any work there..and even if it was, assuming the
~/.Trash was never filled up, it'd not have anything to delete.
> Add a way for an FS instance to say "really, no trash interval at all"
> ----------------------------------------------------------------------
>
> Key: HADOOP-15575
> URL: https://issues.apache.org/jira/browse/HADOOP-15575
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs, fs/s3
> Affects Versions: 3.1.0
> Reporter: Steve Loughran
> Priority: Major
>
> Object stores like S3 often offer [object
> versioning|https://docs.aws.amazon.com/AmazonS3/latest/dev/ObjectVersioning.html]
> as a way to recover from accidental deletions. There is no need to use the
> Fs Shell trash mechanism.
> Proposed: a way for an FS instance to tell the shell to not use trash, even
> if the client has configured it. The current
> {{getServerDefaults().getTrashInterval()}} returns a value from the server,
> but it *only* overrides the ""fs.trash.interval" if it !=0. So if a server
> says "don't use trash" it gets ignored.
> If a special value (-1) is returned (or we use a new field?), FS instances
> can return to the shell saying "no need for trash". Then you can turn it on
> for a bucket by bucket basis; any store with the feature enabled can do the
> same thing.
> Alternative option: A special S3A-aware trash policy which does a
> getFileSystem.getConf.getBoolean("fs.s3a.trash.skip") & skips trash if set.
> This will all you to disable it on a bucket-by-bucket basis.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
