ben-roling opened a new pull request #646: HADOOP-16085: use object version or etags to protect against inconsistent read after replace/overwrite URL: https://github.com/apache/hadoop/pull/646 This started with [HADOOP-16085-003.patch](https://issues.apache.org/jira/secure/attachment/12962649/HADOOP-16085-003.patch) from [the JIRA](https://issues.apache.org/jira/browse/HADOOP-16085). I'm switching over to a PR instead of using patch files attached to the JIRA. I expect that will make review easier. I've addressed a few things since that patch: * copy exception handling - handling 412 error on the response * addressed [Gabor's comments](https://issues.apache.org/jira/browse/HADOOP-16085?focusedCommentId=16797173&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16797173) on TestPathMetadataDynamoDBTranslation, TestDirListingMetadata * fixed a problem I introduced around inconsistency between PathMetadata.isEmptyDir and the underlying S3AFileStatus.isEmpyDir that was manifesting in failures to clean up files after tests * increased default LocalMetadataStore cache timeout as the low 10 second default was making debugging some failing tests confusing as the outcome would depend on how quickly I went through breakpoints * fixed S3 Select test in ITestS3ARemoteFileChanged and added test for copy/rename * improved documentation I haven't actually run all the tests again since these changes. Also, I think there might be a couple more tests to add or alter. For example, I don't have an explicit integration test yet to read a file that has no ETag or versionId in S3Guard. I'll make another pass through but figured it is worthwhile to post the progress.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
