[
https://issues.apache.org/jira/browse/HADOOP-18501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17629776#comment-17629776
]
ASF GitHub Bot commented on HADOOP-18501:
-----------------------------------------
pranavsaxena-microsoft commented on code in PR #5109:
URL: https://github.com/apache/hadoop/pull/5109#discussion_r1015338948
##########
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsRestOperation.java:
##########
@@ -311,6 +319,14 @@ private boolean executeHttpOperation(final int retryCount,
LOG.debug("HttpRequestFailure: {}, {}", httpOperation, ex);
}
+ if (ex instanceof SocketException && CONNECTION_RESET.equals(
Review Comment:
Added. No hooks to enable / disable.
##########
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsClientThrottlingIntercept.java:
##########
@@ -89,8 +90,17 @@ static void updateMetrics(AbfsRestOperationType
operationType,
case ReadFile:
String range =
abfsHttpOperation.getConnection().getRequestProperty(HttpHeaderConfigurations.RANGE);
contentLength = getContentLengthIfKnown(range);
- if (contentLength > 0) {
- singleton.readThrottler.addBytesTransferred(contentLength,
+ long bytesToBeAddedInMetric = contentLength;
Review Comment:
Added
> [ABFS]: Partial Read should add to throttling metric
> ----------------------------------------------------
>
> Key: HADOOP-18501
> URL: https://issues.apache.org/jira/browse/HADOOP-18501
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs/azure
> Affects Versions: 3.3.4
> Reporter: Pranav Saxena
> Assignee: Pranav Saxena
> Priority: Minor
> Labels: pull-request-available
>
> Error Description:
> For partial read (due to account backend throttling), the ABFS driver retry
> but doesn't add up in the throttling metrics.
> In case of partial read with connection-reset exception, ABFS driver retry
> for the full request and doesn't add up in throttling metrics.
> Mitigation:
> In case of partial read, Abfs Driver should retry for the remaining bytes and
> it should be added in throttling metrics.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
