ben-roling commented on issue #794: HADOOP-16085: use object version or etags to protect against inconsistent read after replace/overwrite URL: https://github.com/apache/hadoop/pull/794#issuecomment-491047554 Good catches. This retry stuff is a bit tricky. I obviously overlooked the fact that getObjectMetadata() was retrying. That also needs to be considered in S3AFileSystem.select() where I added a layer of retries to wait for consistency, but it is wrapping getObjectMetadata() as well. A getObjectMetadata() that takes the ChangeTracker as you suggest seems like it could apply in both copyFile() and select(). It seems like a good approach to try. I assume maybe you are already taking a shot at that? If not, I can give it a whirl.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
