[jira] [Commented] (HADOOP-16370) S3AFileSystem copyFile to propagate etag/version from getObjectMetadata to copy request
[ https://issues.apache.org/jira/browse/HADOOP-16370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17469429#comment-17469429 ] Steve Loughran commented on HADOOP-16370: - this came in with hadoop-16805 > S3AFileSystem copyFile to propagate etag/version from getObjectMetadata to > copy request > --- > > Key: HADOOP-16370 > URL: https://issues.apache.org/jira/browse/HADOOP-16370 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 3.3.0 >Reporter: Steve Loughran >Priority: Minor > > Something to consider if we want: should the etag and version from the > initial getObjectMetadata call be propagated to the actual CopyRequest *if > they are not already known* > That way, if we rename() a file and its etag/version is not known, we can fix > them for the next stage of the operation. Relevant given we are copying > metadata over, and for resilience to changes while the copy is taking place -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16370) S3AFileSystem copyFile to propagate etag/version from getObjectMetadata to copy request
[ https://issues.apache.org/jira/browse/HADOOP-16370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16863273#comment-16863273 ] Steve Loughran commented on HADOOP-16370: - +[~ben.roling] Note that when we do a LIST operation against S3, we can collect the etags, but *not* the version info, so a bulk directory rename will not have version markers when the COPY is initiated, just etags. We could use the metadata to actually go versioned > S3AFileSystem copyFile to propagate etag/version from getObjectMetadata to > copy request > --- > > Key: HADOOP-16370 > URL: https://issues.apache.org/jira/browse/HADOOP-16370 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 3.3.0 >Reporter: Steve Loughran >Priority: Minor > > Something to consider if we want: should the etag and version from the > initial getObjectMetadata call be propagated to the actual CopyRequest *if > they are not already known* > That way, if we rename() a file and its etag/version is not known, we can fix > them for the next stage of the operation. Relevant given we are copying > metadata over, and for resilience to changes while the copy is taking place -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
