anmolanmol1234 commented on code in PR #4039:
URL: https://github.com/apache/hadoop/pull/4039#discussion_r1040936094
##########
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsHttpOperation.java:
##########
@@ -314,18 +314,21 @@ public void sendRequest(byte[] buffer, int offset, int
length) throws IOExceptio
if (this.isTraceEnabled) {
startTime = System.nanoTime();
}
- try (OutputStream outputStream = this.connection.getOutputStream()) {
- // update bytes sent before they are sent so we may observe
- // attempted sends as well as successful sends via the
- // accompanying statusCode
- this.bytesSent = length;
+ OutputStream outputStream;
+ try {
+ try {
+ outputStream = this.connection.getOutputStream();
+ } catch (IOException e) {
+ // If getOutputStream fails with an exception due to 100-continue
Review Comment:
1. The first point is valid, I have made the change where getOutputStream
throws exception for the cases where 100 continue is not enabled and returns
back to the caller when it catches an IOException due to 100 continue being
enabled which leads to processResponse getting the correct status code and then
eventually the retry logic coming into play.
2. We need to update the bytes sent for failed as well as passed cases. The
current change will not swallow any exceptions.
The handling for various status code with 100 continue enabled is as follows
1. Case 1 :- getOutputStream doesn't throw any exception, response is
processed and it gives status code of 200, no retry is needed and hence the
request succeeds.
2. Case 2:- getOutputSteam throws exception, we return to the caller and in
processResponse in this.connection.getResponseCode() it gives status code of
404 (user error), exponential retry is not needed. We retry without 100
continue enabled.
3. Case 3:- getOutputSteam throws exception, we return to the caller and in
processResponse it gives status code of 503,
which shows throttling so we backoff accordingly with exponential retry.
Since each append request waits for 100 continue response, the stress on the
server gets reduced.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
