steveloughran commented on a change in pull request #794: HADOOP-16085: use
object version or etags to protect against inconsistent read after
replace/overwrite
URL: https://github.com/apache/hadoop/pull/794#discussion_r285060816
##########
File path:
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/Listing.java
##########
@@ -390,8 +392,18 @@ public S3AFileStatus next() throws IOException {
status = statusBatchIterator.next();
// We remove from provided list the file status listed by S3 so that
// this does not return duplicate items.
- if (providedStatus.remove(status)) {
- LOG.debug("Removed the status from provided file status {}", status);
+
+ // The provided status is returned as it is assumed to have the better
+ // metadata (i.e. the eTag and versionId from S3Guard)
+ Optional<S3AFileStatus> provided =
Review comment:
OK, I see now. Set.remove() doesn't return the old version, only that there
was an old version
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
