steveloughran commented on issue #575: HADOOP-13327 Output Stream Specification URL: https://github.com/apache/hadoop/pull/575#issuecomment-548815620 Had some time to look at the comments here after too long break; more work is needed ## Output stream spec Needs more review of the comments, especiallys Sean's ## Stream Model + impl. I now suspect that I need to do a bit more resilience in the failure handling of S3ABlockOutputStream. Specifically: if 1+ POST has failed in a multipart write, we should make a best effort attempt to abort the upload. Admittedly, the same network or authentication errors which cause the post to fail are likely to cause the abort operation to fail too -that should not stop us up from at least trying. Proposed: close() will attempt to abort the upload if a failure occurred earlier. This is going to be fun to test. Indeed, it's a good argument for WriteOperationHelper to be converted to an interface with a mocking implementation alongside the production one. I might play with some subclassing instead; ## Plan I think these two bits of work can be separated; the S3ABlockOutput stream model/locking/aborting is complex enough to merit isolation. Will follow up with that in a separate PR while retaining this purely for the output stream spec and tests. This one doesn't apply to trunk no more; once I have gone through the comments I may produce a new PR for the output stream changes too.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
