[
https://issues.apache.org/jira/browse/HADOOP-13892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16156736#comment-16156736
]
Steve Loughran commented on HADOOP-13892:
-----------------------------------------
S3 tags are (a) not broadly supported and (b) limited to 10 entries of 256 each.
There's also HTTP headers set up front, which have 2048 bytes of data.
To work fully with distcp, you need to support get & set xattr, which is
critical to back up encrypted HDFS data...this is only being set after the
output stream is created. Maybe it could be added to the builder API so that
perms are set at create time
> use s3 to record permissions on objects, so preserving them through distcp
> round trips
> --------------------------------------------------------------------------------------
>
> Key: HADOOP-13892
> URL: https://issues.apache.org/jira/browse/HADOOP-13892
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 2.9.0
> Reporter: Steve Loughran
>
> S3 now supports object tags, attributes which can be updated during the life
> of an object.
> S3A could use that to preserve the permissions/ACLs of objects when copied
> from elsewhere, in particular from HDFS. This would ensure that data backed
> up from HDFS preserves all the permission information needed when doing a
> recovery from S3 to HDFS.
> Azure WASB does exactly this already.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
