noslowerdna commented on a change in pull request #646: HADOOP-16085: use object version or etags to protect against inconsistent read after replace/overwrite URL: https://github.com/apache/hadoop/pull/646#discussion_r269753654
########## File path: hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/s3guard.md ########## @@ -1021,6 +1117,41 @@ java.io.IOException: Invalid region specified "iceland-2": The region specified in `fs.s3a.s3guard.ddb.region` is invalid. +### Error `RemoteFileChangedException` + +An exception like the following could occur for a couple of reasons: + +* the S3Guard metadata is out of sync with the true S3 metadata. For +example, the S3Guard DynamoDB table is tracking a different ETag than the ETag +shown in the exception. This may suggest the object was updated in S3 without +involvement from S3Guard or there was a transient failure when S3Guard tried to +write to S3. + +* S3 is exhibiting read-after-overwrite eventual consistency. The S3Guard Review comment: Might be more clear to say "temporary inconsistency" instead of "eventual consistency" here? ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
