ben-roling commented on a change in pull request #646: HADOOP-16085: use object
version or etags to protect against inconsistent read after replace/overwrite
URL: https://github.com/apache/hadoop/pull/646#discussion_r269762117
##########
File path:
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/ChangeTracker.java
##########
@@ -148,16 +177,77 @@ public void processResponse(final S3Object object,
}
}
- final ObjectMetadata metadata = object.getObjectMetadata();
+ processMetadata(object.getObjectMetadata(), operation, pos);
+ }
+
+ /**
+ * Process the response from the server for validation against the
+ * change policy.
+ * @param copyResult result of a copy operation
+ * @throws PathIOException raised on failure
+ * @throws RemoteFileChangedException if the remote file has changed.
+ */
+ public void processResponse(final CopyResult copyResult)
+ throws PathIOException {
+ // ETag (sometimes, depending on encryption and/or multipart) is not the
+ // same on the copied object as the original. Version Id seems to never
+ // be the same on the copy. As such, there isn't really anything that
+ // can be verified on the response.
+ }
+
+ /**
+ * Process an exception generated against the change policy.
+ * If the exception indicates the file has changed, this method throws
+ * {@code RemoteFileChangedException} with the original exception as the
+ * cause.
+ * @param e the exception
+ * @param operation the operation performed when the exception was
+ * generated.
+ * @throws RemoteFileChangedException if the remote file has changed.
+ */
+ public void processException(Exception e, String operation) throws
+ RemoteFileChangedException {
+ if (e instanceof AmazonServiceException) {
+ AmazonServiceException serviceException = (AmazonServiceException) e;
+ if (serviceException.getStatusCode() == 412) {
+ versionMismatches.incrementAndGet();
+ throw new RemoteFileChangedException(uri, operation, String.format(
+ RemoteFileChangedException.PRECONDITIONS_NOT_MET
+ + " on %s."
+ + " Version %s was unavailable",
+ getSource(),
Review comment:
The only operation string currently going through here is "copy". It does
not contain the path. Source is "etag" or "versionid". The path is included
in the exception via the uri provided in the constructor which flows down to
the base PathIOException.
If the "read" or "select" paths threw 412 or 412-like exceptions they could
flow through here too, but they do not. Read returns a null S3Object, which
flows into processResponse(). Select doesn't directly support ETag or
versionId qualification so that code path is making a call to processMetadata()
before the read.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
