ben-roling commented on a change in pull request #606: HADOOP-16190. S3A
copyFile operation to include source versionID or etag in the copy request
URL: https://github.com/apache/hadoop/pull/606#discussion_r268790530
##########
File path:
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java
##########
@@ -2888,12 +2890,19 @@ private void copyFile(String srcKey, String dstKey,
long size)
setOptionalCopyObjectRequestParameters(copyObjectRequest);
copyObjectRequest.setCannedAccessControlList(cannedACL);
copyObjectRequest.setNewObjectMetadata(dstom);
+ String id = srcom.getVersionId();
+ if (id != null) {
Review comment:
The way I have been approaching this in my work for HADOOP-16085 is to use
ChangeDetectionPolicy and ChangeTracker. I was adding new
ChangeTracker.maybeApplyConstraint(CopyObjectRequest) and
ChangeDetectionPolicy.applyRevisionConstraint(CopyObjectRequest, revisionId)
methods to support that.
I call ChangeTracker.processMetadata(srcom) and then
ChangeTracker.maybeApplyConstraint(copyObjectRequest). Resulting behavior
depends on ChangeDetectionPolicy, but would generally be similar to what you
are doing here. A couple of differences:
* if change.detection.mode=none, my code doesn't add the constraint
* if change.detection.version.required=true, my code will throw
NoVersionAttributeException when the attribute defined by
change.detection.source is unavailable
What do you think about that?
Side note: my branch doesn't currently handle exceptions on copy (the 412
error condition) properly. This PR helped remind me of that.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
