steveloughran commented on a change in pull request #666: HADOOP-16221 add
option to fail operation on metadata write failure
URL: https://github.com/apache/hadoop/pull/666#discussion_r270537092
##########
File path:
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java
##########
@@ -3034,10 +3042,13 @@ private void setOptionalObjectMetadata(ObjectMetadata
metadata) {
* </ol>
* @param key key written to
* @param length total length of file written
+ * @throws IOException if metadata about the write could
+ * not be saved to the metadata store
*/
@InterfaceAudience.Private
- @Retries.RetryExceptionsSwallowed
- void finishedWrite(String key, long length) {
+ @Retries.OnceRaw
Review comment:
DynamoDB metastore putAndReturn is Retrydoes retry; what's changing now is
that
raw S3 FS Delete exceptions are being swallowed; when eventually s3guard
gives up, that's thrown. Afraid you are going to have to chase through that
call chain to see what's going on and make sure things are consistent w.r.t
retry/translation and with clarifications. If there's some stuff in between
which isn't fully annotated (putAndReturn()), this is the chance to fix that. I
know it's a pain, but trying to wrap retry() with retry() causes an exponential
explosion in the time a failure takes to surface and we need to keep absolutely
on top of that (more than the translated/raw stuff)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
