[
https://issues.apache.org/jira/browse/HADOOP-18501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17629779#comment-17629779
]
ASF GitHub Bot commented on HADOOP-18501:
-----------------------------------------
pranavsaxena-microsoft commented on code in PR #5109:
URL: https://github.com/apache/hadoop/pull/5109#discussion_r1015339894
##########
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/services/MockAbfsClientThrottlingAnalyzer.java:
##########
@@ -0,0 +1,49 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ * <p>
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * <p>
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.fs.azurebfs.services;
+
+public class MockAbfsClientThrottlingAnalyzer
+ extends AbfsClientThrottlingAnalyzer {
+
+ private ThreadLocal<Integer> failedInstances = new ThreadLocal<>();
+
+ public MockAbfsClientThrottlingAnalyzer(final String name)
+ throws IllegalArgumentException {
+ super(name);
+ }
+
+ @Override
+ public void addBytesTransferred(final long count,
+ final boolean isFailedOperation) {
+ if (isFailedOperation) {
+ Integer current = failedInstances.get();
+ if (current == null) {
+ current = 1;
+ } else {
+ current++;
+ }
+ failedInstances.set(current);
+ }
+ super.addBytesTransferred(count, isFailedOperation);
+ }
+
+ public Integer getFailedInstances() {
+ return failedInstances.get();
+ }
+}
Review Comment:
Refactored
> [ABFS]: Partial Read should add to throttling metric
> ----------------------------------------------------
>
> Key: HADOOP-18501
> URL: https://issues.apache.org/jira/browse/HADOOP-18501
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs/azure
> Affects Versions: 3.3.4
> Reporter: Pranav Saxena
> Assignee: Pranav Saxena
> Priority: Minor
> Labels: pull-request-available
>
> Error Description:
> For partial read (due to account backend throttling), the ABFS driver retry
> but doesn't add up in the throttling metrics.
> In case of partial read with connection-reset exception, ABFS driver retry
> for the full request and doesn't add up in throttling metrics.
> Mitigation:
> In case of partial read, Abfs Driver should retry for the remaining bytes and
> it should be added in throttling metrics.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
