ben-roling commented on a change in pull request #646: HADOOP-16085: use object
version or etags to protect against inconsistent read after replace/overwrite
URL: https://github.com/apache/hadoop/pull/646#discussion_r269602376
##########
File path:
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java
##########
@@ -3490,52 +3607,55 @@ public EtagChecksum getFileChecksum(Path f, final long
length)
entryPoint(INVOCATION_LIST_LOCATED_STATUS);
Path path = qualify(f);
LOG.debug("listLocatedStatus({}, {}", path, filter);
- return once("listLocatedStatus", path.toString(),
- () -> {
- // lookup dir triggers existence check
- final FileStatus fileStatus = getFileStatus(path);
- if (fileStatus.isFile()) {
- // simple case: File
- LOG.debug("Path is a file");
- return new Listing.SingleStatusRemoteIterator(
- filter.accept(path) ? toLocatedFileStatus(fileStatus) : null);
- } else {
- // directory: trigger a lookup
- final String key = maybeAddTrailingSlash(pathToKey(path));
- final Listing.FileStatusAcceptor acceptor =
- new Listing.AcceptAllButSelfAndS3nDirs(path);
- DirListingMetadata meta =
- S3Guard.listChildrenWithTtl(metadataStore, path,
- ttlTimeProvider);
- final RemoteIterator<FileStatus> cachedFileStatusIterator =
- listing.createProvidedFileStatusIterator(
- S3Guard.dirMetaToStatuses(meta), filter, acceptor);
- return (allowAuthoritative && meta != null
- && meta.isAuthoritative())
- ? listing.createLocatedFileStatusIterator(
- cachedFileStatusIterator)
- : listing.createLocatedFileStatusIterator(
- listing.createFileStatusListingIterator(path,
- createListObjectsRequest(key, "/"),
- filter,
- acceptor,
- cachedFileStatusIterator));
- }
- });
+ RemoteIterator<? extends LocatedFileStatus> iterator =
+ once("listLocatedStatus", path.toString(),
Review comment:
The GitHub diff makes this look like a bigger change than it is due to
change in indentation. Really all that changed in this block is assignment of
the once() return value to the iterator on line 3610, the call to
toLocatedFileStatusIterator() on 3644, and use of S3AFileStatus instead of
vanilla FileStatus on lines 3614, 3629.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
