TSFenwick commented on code in PR #14131:
URL: https://github.com/apache/druid/pull/14131#discussion_r1272777356
##########
extensions-core/s3-extensions/src/main/java/org/apache/druid/storage/s3/S3DataSegmentKiller.java:
##########
@@ -64,13 +74,116 @@ public S3DataSegmentKiller(
this.inputDataConfig = inputDataConfig;
}
+ @Override
+ public void kill(List<DataSegment> segments) throws SegmentLoadingException
+ {
+ if (segments.isEmpty()) {
+ return;
+ }
+ if (segments.size() == 1) {
+ kill(segments.get(0));
+ return;
+ }
+
+ // create a map of bucket to keys to delete
+ Map<String, List<DeleteObjectsRequest.KeyVersion>> bucketToKeysToDelete =
new HashMap<>();
+ for (DataSegment segment : segments) {
+ String s3Bucket = MapUtils.getString(segment.getLoadSpec(),
S3DataSegmentPuller.BUCKET);
+ String path = MapUtils.getString(segment.getLoadSpec(),
S3DataSegmentPuller.KEY);
+ List<DeleteObjectsRequest.KeyVersion> keysToDelete = new ArrayList<>();
+ keysToDelete.add(new DeleteObjectsRequest.KeyVersion(path));
+ keysToDelete.add(new
DeleteObjectsRequest.KeyVersion(DataSegmentKiller.descriptorPath(path)));
+ bucketToKeysToDelete.computeIfAbsent(s3Bucket, k -> new
ArrayList<>()).addAll(keysToDelete);
Review Comment:
good find
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
