steveloughran commented on issue #794: HADOOP-16085: use object version or etags to protect against inconsistent read after replace/overwrite URL: https://github.com/apache/hadoop/pull/794#issuecomment-491013417 That'd be great. If you turn @debug logging on for the tests you can get a better view of what's happening in terms of when an operation is being called. Now some other issues 1. if the versionId is known, should it be included in the `getObjectMetadata()` Call? Right now it's being checked on the client, but `GetObjectMetadataRequest` does take a version ID which could be passed in 2. the copy operation has moved from once() to retry(), something I've updated in the @Retry tag. But: * getObjectMetdata() Already does retry: it shouldn't be wrapped in case of extended connection failures amplifying delays * I Believe the transfer manager is meant to do its own recovery. (but I've also known it to fail once with a 200 + Error) state I'm worried that the addition of retry resilience may be putting a double wrapper around existing failure conditions and their retries. * Given that `getObjectMetadata()` only needs the new retry to handle its eventualness, if we passed the ChangeTracker into that method, the existing retry-invoked operation could both set the version ID and check on the client side for problems. It would then be pulled out into its own once() call just to translate exceptions. * And for the transfer, we could have it retrying with a (new) retry policy which only retries on 412 responses, so any other failure doesn't trigger a retry. BTW, note that the CopyOutcome class only stores SdkBaseExceptions raised in the copy, as does the method to await the copy to complete. I've not put that through all the way through the other bits of the patch, but think it makes sense as other failures shouldn't need to be considered by the change tracker.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
