ahmarsuhail commented on code in PR #5995:
URL: https://github.com/apache/hadoop/pull/5995#discussion_r1308825152
##########
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3ARetryPolicy.java:
##########
@@ -139,6 +144,9 @@ public S3ARetryPolicy(Configuration conf) {
retryIdempotentCalls = new FailNonIOEs(
new IdempotencyRetryFilter(baseExponentialRetry));
+ //retryAwsClientExceptions = new
RetryFromAWSClientIOException(retryIdempotentCalls);
+ retryAwsClientExceptions = retryIdempotentCalls;
Review Comment:
nit: add a TODO here so it's clear why this commented out. confused me till
I read the comment on the commit
##########
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3ARetryPolicy.java:
##########
@@ -224,9 +232,11 @@ protected Map<Class<? extends Exception>, RetryPolicy>
createExceptionMap() {
// server didn't respond.
policyMap.put(AWSNoResponseException.class, retryIdempotentCalls);
+ // translated AWS exceptions are retried if retryable() is true.
+ policyMap.put(AWSClientIOException.class, retryAwsClientExceptions);
+ policyMap.put(AWSServiceIOException.class, retryAwsClientExceptions);
Review Comment:
when we do switch to the new retry policy, doesn't this mean
AWSServiceIOException will not be retried? I think previously these were
retried, but not sure
##########
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/AWSCredentialProviderList.java:
##########
@@ -330,4 +332,27 @@ public void close() {
public int size() {
return providers.size();
}
+
+
+ /**
+ * Translate an exception if it or its inner exception is an
+ * {@link CredentialInitializationException}.
+ * If this condition is not met, null is returned.
+ * @param path path of operation.
+ * @param throwable exception
+ * @return a translated exception or null.
+ */
+ public static IOException maybeTranslateCredentialException(String path,
Review Comment:
why do we need to do this? I get the AuditFailures are getting wrapped in a
SDKClient exception, what's wrapping CredentialInitializationException?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
