clintropolis commented on a change in pull request #9858:
URL: https://github.com/apache/druid/pull/9858#discussion_r424797480
##########
File path:
extensions-core/s3-extensions/src/main/java/org/apache/druid/storage/s3/S3InputDataConfig.java
##########
@@ -29,16 +32,33 @@
*/
public class S3InputDataConfig
{
+ @VisibleForTesting
+ static final int MAX_LISTING_LENGTH_MIN = 1;
+
+ /**
+ * AWS S3 only allows deleting 1000 elements at a time:
+ *
https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/s3/model/DeleteObjectsRequest.html
Review comment:
It's not super clear whether or not list objects API is limited on the
server side to a max of 1000, From
https://docs.aws.amazon.com/AmazonS3/latest/API/API_ListObjects.html#API_ListObjects_RequestSyntax
>Sets the maximum number of keys returned in the response. By default the
API returns up to 1,000 key names. The response might contain fewer keys but
will never contain more.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
