[
https://issues.apache.org/jira/browse/HADOOP-14971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16220474#comment-16220474
]
ASF GitHub Bot commented on HADOOP-14971:
-----------------------------------------
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/hadoop/pull/282#discussion_r147152407
--- Diff:
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/InconsistentAmazonS3Client.java
---
@@ -426,8 +490,10 @@ private void registerDeleteObject(String key, String
bucket) {
if (shouldDelay(key)) {
// Record summary so we can add it back for some time post-deletion
S3ObjectSummary summary = null;
- ObjectListing list = listObjects(bucket, key);
- for (S3ObjectSummary result : list.getObjectSummaries()) {
+ ListObjectsV2Request request =
+ new ListObjectsV2Request().withBucketName(bucket)
+ .withPrefix(key);
--- End diff --
No real reason. I do recall how an aggressive throttle rate had real
problems with listing because a single op could fail too many times for the
retry handlers to cope. But I don't see that this has fixed it because its not
calling the innerList() op which is needed to bypass that inconsistency.
I'll revert to v2 and innerListV1, so that there's no failures in this
loop, and, assuming all stores support v1 alongside v2, work everywher
> Merge S3A committers into trunk
> -------------------------------
>
> Key: HADOOP-14971
> URL: https://issues.apache.org/jira/browse/HADOOP-14971
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 3.0.0
> Reporter: Steve Loughran
> Assignee: Steve Loughran
>
> Merge the HADOOP-13786 committer into trunk. This branch is being set up as a
> github PR for review there & to keep it out the mailboxes of the watchers on
> the main JIRA
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
