steveloughran commented on a change in pull request #628: HADOOP-16186. NPE in
ITestS3AFileSystemContract teardown in DynamoDBMetadataStore.lambda$listChildren
URL: https://github.com/apache/hadoop/pull/628#discussion_r269117119
##########
File path:
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/s3guard/DynamoDBMetadataStore.java
##########
@@ -638,22 +638,42 @@ public DirListingMetadata listChildren(final Path path)
throws IOException {
metas.add(meta);
}
+ // Minor race condition here - if the path is deleted between
+ // getting the list of items and the directory metadata we might
+ // get a null in DDBPathMetadata.
DDBPathMetadata dirPathMeta = get(path);
- boolean isAuthoritative = false;
- if(dirPathMeta != null) {
- isAuthoritative = dirPathMeta.isAuthoritativeDir();
- }
-
- LOG.trace("Listing table {} in region {} for {} returning {}",
- tableName, region, path, metas);
- return (metas.isEmpty() && dirPathMeta == null)
- ? null
- : new DirListingMetadata(path, metas, isAuthoritative,
- dirPathMeta.getLastUpdated());
+ return getDirListingMetadataFromDirMetaAndList(path, metas,
+ dirPathMeta);
});
}
+ DirListingMetadata getDirListingMetadataFromDirMetaAndList(Path path,
+ List<PathMetadata> metas, DDBPathMetadata dirPathMeta) {
+ boolean isAuthoritative = false;
+ if(dirPathMeta != null) {
+ isAuthoritative = dirPathMeta.isAuthoritativeDir();
+ }
+
+ LOG.trace("Listing table {} in region {} for {} returning {}",
+ tableName, region, path, metas);
+
+ if (!metas.isEmpty() && dirPathMeta == null) {
+ // We handle this case as the directory is deleted.
+ LOG.warn("Directory marker is deleted, but the list of the directory "
+ + "elements is not empty: {}. This case is handled as the directory"
Review comment:
"is handled as if"?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
