zachjsh commented on a change in pull request #9858:
URL: https://github.com/apache/druid/pull/9858#discussion_r424800972
##########
File path:
extensions-core/s3-extensions/src/main/java/org/apache/druid/storage/s3/S3InputDataConfig.java
##########
@@ -29,16 +32,33 @@
*/
public class S3InputDataConfig
{
+ @VisibleForTesting
+ static final int MAX_LISTING_LENGTH_MIN = 1;
+
+ /**
+ * AWS S3 only allows deleting 1000 elements at a time:
+ *
https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/s3/model/DeleteObjectsRequest.html
Review comment:
Yeah I think we should leave both listing and deleting to max of 1000
keys. I can add the link to this documentation in the javadocs as well. Let me
know if this sounds reasonable to you.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
